Orchestrating privacy-protected big data analyses of data from different resources with R and DataSHIELD

Area of Science:

• Bioinformatics
• Computational Biology
• Data Science

Background:

• Combined analysis of large datasets is crucial in health and biosciences.
• Existing methods face challenges with data transfer, ethico-legal constraints, and analytical inflexibility.
• The DataSHIELD platform facilitates privacy-preserving data analysis by keeping data at its source.

Purpose of the Study:

• To present a new architecture for DataSHIELD and Opal to enable the analysis of large, complex datasets.
• To overcome limitations in storage formats and analytical capabilities of previous DataSHIELD versions.
• To facilitate privacy-preserving big data analysis in genomics and geospatial projects.

Main Methods:

• Development of a new "resources" architecture for DataSHIELD and Opal.
• Integration of external computing facilities for data analysis.
• Extension of the resources concept to support big data infrastructures like GA4GH and EGA.
• Utilization of shell commands for advanced data manipulation.

Main Results:

• The new architecture allows large, complex datasets to be analyzed in their original location and format.
• Demonstrated real-world big data analysis examples in genomics and geospatial projects.
• Successfully extended the resources concept to address specific big data infrastructures.
• Enabled privacy-preserving data analysis from existing data sharing initiatives.

Conclusions:

• The new architecture significantly enhances DataSHIELD's capability for big data analysis.
• It overcomes previous limitations, allowing for more flexible and powerful privacy-preserving analyses.
• This framework supports researchers in conducting complex analyses on sensitive data without compromising privacy or data integrity.