Integrating medical and research information: a big data approach

Area of Science:

• Biomedical Informatics
• Data Science
• Business Intelligence

Background:

• Instituto de Investigación Biomédica de A Coruña (INIBIC) faces challenges managing high-volume, heterogeneous, unstructured research data.
• Integrating this data into medical information systems requires specialized data collection and organization architectures.
• Existing data management strategies are insufficient for the scale and complexity of INIBIC's research information.

Purpose of the Study:

• To introduce the Hadoop framework as a solution for integrating unstructured research data within the Business Intelligence domain.
• To demonstrate Hadoop's capability in collecting, exploring, processing, and structuring diverse research information.
• To establish a framework for creating a data mart equivalent for Business Intelligence systems using research data.

Main Methods:

• Implementation of the Hadoop framework for large-scale data processing.
• Utilizing Hadoop's distributed computing capabilities to handle data volume and heterogeneity.
• Developing data processing pipelines to structure unstructured research information for analysis.

Main Results:

• Successful application of the Hadoop framework to manage and organize INIBIC's unstructured research data.
• Demonstrated ability of Hadoop to collect, explore, process, and structure diverse datasets.
• Development of a functional equivalent to a data mart within a Business Intelligence system.

Conclusions:

• The Hadoop framework provides a robust solution for integrating and managing complex biomedical research data.
• This approach facilitates the analysis of previously inaccessible unstructured information, enhancing Business Intelligence capabilities.
• Hadoop enables the creation of valuable data marts from research data, supporting advanced medical information systems.