Prediction model optimization using full model selection with regression trees demonstrated with FTIR data from...

Area of Science:

• Veterinary Medicine
• Data Science
• Machine Learning

Background:

• Predictive modeling in epidemiology often relies on empirical selection of data preprocessing and algorithms, which can impact model performance.
• Existing full model selection (FMS) methods have limitations, including dependency on user-selected hyperparameters, hindering routine use for model optimization.

Purpose of the Study:

• To introduce and evaluate regression tree full model selection (rtFMS) as an innovative and systematic approach for optimizing predictive modeling.
• To develop milk-based prediction models for nonesterified fatty acids (NEFA) and β-hydroxybutyrate acid (BHBA) for dairy cow metabolic health screening.

Main Methods:

• rtFMS involves developing models for all considered predictive modeling option combinations.
• Iterated, cross-validation performances are processed through a regression tree to select the optimal final model.
• Applied to a milk Fourier transform infrared spectroscopy dataset to predict NEFA and BHBA levels in dairy cows.

Main Results:

• rtFMS provides a non-black box, interpretable, and hyperparameter-free method for model selection.
• The method illustrates the relative importance of different modeling options and eliminates user bias.
• Demonstrated successful application in building predictive models for dairy cow metabolic health parameters.

Conclusions:

• rtFMS offers significant advantages over empirical selection of preprocessing and algorithms for predictive modeling.
• The developed milk-based models can serve as a screening tool for dairy cattle metabolic health.
• rtFMS facilitates systematic and unbiased optimization of predictive models in various applications.