Implementation of a Big Data Accessing and Processing Platform for Medical Records in Cloud

Area of Science:

• Computer Science
• Health Informatics
• Data Science

Background:

• Increasing medical data volume and complexity pose challenges for analysis.
• Scattered and varied data formats hinder integration and utilization of medical records.
• Scalable cloud platforms are essential for managing and analyzing large medical datasets.

Purpose of the Study:

• To develop a cloud storage system for big medical data analysis using Hadoop HBase.
• To improve the performance of importing medical records into a database.
• To provide functionalities for searching, filtering, and analyzing medical data.

Main Methods:

• Implementation of a cloud storage system utilizing Hadoop HBase.
• Application of Hadoop MapReduce for distributed processing of medical records.
• Employing single-threaded Put and CompleteBulkload mechanisms for data import optimization.
• Development of a web interface for user interaction and data analysis.

Main Results:

• The system successfully stores and analyzes big medical data in HBase.
• Optimized data import strategies were identified: single-threaded Put for <300MB files and CompleteBulkload for >300MB files.
• Experimental results demonstrate improved data import performance.
• The web interface enables effective data searching, filtering, and analysis.

Conclusions:

• The developed cloud system effectively manages and analyzes big medical data.
• Optimized data import methods significantly enhance system performance.
• The system provides valuable tools for medical professionals to derive insights from data.