An Efficient Data Partitioning to Improve Classification Performance While Keeping Parameters Interpretable.

Area of Science:

• Machine Learning
• Computational Neuroscience
• Statistical Modeling

Background:

• Supervised machine learning commonly uses separate data splits for training, validation, and testing.
• Cross-validation is standard for parameter tuning but can be inefficient with limited data.
• Existing methods face a trade-off between statistical power and model optimization when data is scarce.

Purpose of the Study:

• To introduce a novel machine learning approach called "Cross-validation and cross-testing" (CVCT).
• To improve the trade-off between generalization performance estimation and model fitting with limited data.
• To validate the CVCT approach using simulated and real-world electrophysiological data.

Main Methods:

• Implementation of the proposed "Cross-validation and cross-testing" (CVCT) method.
• Validation using simulated datasets.
• Application to human and rodent electrophysiological recordings.

Main Results:

• The CVCT approach demonstrated a higher probability of discovering significant results compared to standard cross-validation and testing.
• The method maintained the nominal alpha level, ensuring statistical validity.
• CVCT preserves parameter interpretability, unlike nested cross-validation.

Conclusions:

• "Cross-validation and cross-testing" offers an improved strategy for machine learning with limited data.
• The approach enhances statistical power without compromising parameter interpretability.
• CVCT is particularly beneficial when model parameter interpretability is crucial.