Multivariate survival analysis in big data: A divide-and-combine approach

Area of Science:

• Biostatistics
• Health Informatics
• Epidemiology

Background:

• Multivariate failure time data analysis commonly uses marginal proportional hazards or frailty models.
• Extremely large datasets present significant computational challenges for traditional analysis methods.

Purpose of the Study:

• To propose a divide-and-combine method for analyzing large-scale multivariate failure time data.
• To address computational limitations encountered with massive datasets.

Main Methods:

• A divide-and-combine strategy is proposed, involving random data subsetting and weighted estimator combination.
• Regularized estimation is applied to the combined estimator for risk factor screening.
• Theoretical properties including consistency and oracle properties are investigated.

Main Results:

• The proposed combined estimator is asymptotically equivalent to the full data estimator.
• The method effectively handles large-scale data, as demonstrated by application to the Myocardial Infarction Data Acquisition System (MIDAS) dataset.
• Simulation studies confirm the method's performance.

Conclusions:

• The divide-and-combine method provides a computationally efficient and statistically sound approach for large-scale multivariate failure time data analysis.
• This method facilitates the identification of risk factors for complex health outcomes in massive datasets.