Controlled feature selection and compressive big data analytics: Applications to biomedical and health studies

Area of Science:

• Big Data Science
• Computational Statistics
• Bioinformatics

Background:

• Theoretical foundations for Big Data Science remain underdeveloped.
• Existing methods struggle with complex, incomplete, and multi-source datasets.
• Scalable and robust inference frameworks are needed for high-throughput analytics.

Purpose of the Study:

• Propose a new scalable framework for Big Data representation and analytics.
• Develop a model-free inference approach for Big Data.
• Address challenges in handling complex, incongruent, incomplete, and multi-source data.

Main Methods:

• Introduced Compressive Big Data analytics (CBDA), a novel framework.
• CBDA employs iterative random subsampling (feature and case levels) with replacement.
• Utilizes an ensemble predictor applying inference techniques to harmonized samples.

Main Results:

• CBDA enables high-throughput analytics, including variable selection and noise reduction.
• Derived likelihoods and probabilities assess algorithm reliability via bootstrapping.
• Demonstrated scalability and validated on simulated and real neuroimaging-genetics data.

Conclusions:

• CBDA offers a scalable solution for scientific inference with large, complex datasets.
• The framework supports generic representation of multimodal datasets.
• Further mathematical framework development will enable study of ergodic properties and asymptotics.