Efficient approximate k-fold and leave-one-out cross-validation for ridge regression

Area of Science:

• Statistics
• Computational Biology
• Bioinformatics

Background:

• Cross-validation is a standard resampling technique for model evaluation.
• Traditional cross-validation is computationally intensive, particularly for large datasets.
• Generalized linear models and Cox's proportional hazards model are widely used in statistical analysis.

Purpose of the Study:

• To present a computationally efficient approximation method for cross-validation.
• To enable faster model tuning and evaluation in generalized linear and Cox models with ridge penalties.
• To reduce the time required for selecting optimal tuning parameters.

Main Methods:

• An approximation method based on Taylor expansion around the full model estimate.
• Avoids refitting the model for each cross-validation fold.
• Applicable to generalized linear models and Cox's proportional hazards model with ridge penalty terms.

Main Results:

• The approximation method significantly reduces computation time compared to traditional cross-validation.
• The method provides accurate approximations, especially for leave-one-out cross-validation on large datasets.
• Demonstrated performance on several microarray datasets.
• An R package 'penalized' is available to implement the method.

Conclusions:

• The proposed approximation method offers a substantial speed-up for cross-validation.
• It enables efficient tuning parameter selection for penalized regression models.
• This approach is particularly beneficial for large-scale genomic and survival data analysis.