A Multi-Dimensional Big Data Storing System for Generated COVID-19 Large-Scale Data using Apache Spark

by   Manar A. Elmeiligy, et al.

The ongoing outbreak of coronavirus disease (COVID-19) had burst out in Wuhan China, specifically in December 2019. COVID-19 has caused by a new virus that had not been identified in human previously. This was followed by a widespread and rapid spread of this epidemic throughout the world. Daily, the number of the confirmed cases are increasing rapidly, number of the suspect increases, based on the symptoms that accompany this disease, and unfortunately number of the deaths also increase. Therefore, with these increases in number of cases around the world, it becomes hard to manage all these cases information with different situations; if the patient either injured or suspect with which symptoms that appeared on the patient. Therefore, there is a critical need to construct a multi-dimensional system to store and analyze the generated large-scale data. In this paper, a Comprehensive Storing System for COVID-19 data using Apache Spark (CSS-COVID) is proposed, to handle and manage the problem caused by increasing the number of COVID-19 daily. CSS-COVID helps in decreasing the processing time for querying and storing COVID-19 daily data. CSS-COVID consists of three stages, namely, inserting and indexing, storing, and querying stage. In the inserting stage, data is divided into subsets and then index each subset separately. The storing stage uses set of storing-nodes to store data, while querying stage is responsible for handling the querying processes. Using Apache Spark in CSS-COVID leverages the performance of dealing with large-scale data of the coronavirus disease injured whom increase daily. A set of experiments are applied, using real COVID-19 Datasets, to prove the efficiency of CSS-COVID in indexing large-scale data.


Scalable Analysis for Covid-19 and Vaccine Data

This paper explains the scalable methods used for extracting and analyzi...

C-Watcher: A Framework for Early Detection of High-Risk Neighborhoods Ahead of COVID-19 Outbreak

The novel coronavirus disease (COVID-19) has crushed daily routines and ...

Forecasting of COVID-19 Cases, Using an Evolutionary Neural Architecture Search Approach

In late 2019, COVID-19, a severe respiratory disease, emerged, and since...

The role of swabs in modeling the COVID-19 outbreak in the most affected regions of Italy

The daily fluctuations in the released number of Covid-19 cases played a...

A Big Data Based Framework for Executing Complex Query Over COVID-19 Datasets (COVID-QF)

COVID-19's rapid global spread has driven innovative tools for Big Data ...

Leveraging Data Preparation, HBase NoSQL Storage, and HiveQL Querying for COVID-19 Big Data Analytics Projects

Epidemiologist, Scientists, Statisticians, Historians, Data engineers an...

Did COVID-19 infections decline before UK lockdown?

The number of new infections per day is a key quantity for effective epi...

Please sign up or login with your details

Forgot password? Click here to reset