A simulation-approximation approach to sample size planning for high-dimensional classification studies.

Area of Science:

• Statistics
• Machine Learning
• Bioinformatics

Background:

• High-dimensional classification studies with limited sample sizes are common.
• Assessing the impact of sample size on study performance is crucial but challenging due to complex pattern discovery methods.

Purpose of the Study:

• To develop an efficient method for estimating generalization error in high-dimensional classification.
• To investigate how study design parameters influence classification performance and validation results.

Main Methods:

• Combines Monte Carlo methods with novel approximations for linear discriminant analysis under multivariate normal distributions.
• Compares Taylor series approximation of generalization error with normal distribution approximation of discriminant scores.
• Utilizes full simulations to evaluate the developed method across various realistic study design scenarios.

Main Results:

• The combined Monte Carlo and approximation approach efficiently estimates expected generalization error.
• Both approximation methods performed well generally, but normal discriminant score approximation excelled with many uninformative features.
• Simulations revealed that many realistic study designs may lead to suboptimal pattern estimation and low statistical validation power.

Conclusions:

• The developed method provides an efficient way to estimate generalization error in high-dimensional classification.
• Study design choices significantly impact classification performance, with potential for suboptimal results and low statistical significance in practice.
• Careful consideration of sample size, feature informativeness, and feature selection is critical for robust high-dimensional classification studies.