Prediction of driving ability: Are we building valid models?

Area of Science:

• Driving research
• Psychometrics
• Statistical modeling

Background:

• Predicting on-road driving ability from off-road measures is a key research objective.
• Classification models often suffer from over-fitting, leading to inflated accuracy and poor generalization.
• Many studies lack sufficient detail to assess over-fitting risk or report validation techniques.

Purpose of the Study:

• To investigate the impact of sample size and variable ratios on model over-fitting in driving research.
• To identify best practices for developing accurate and generalizable predictive models of driving ability.
• To provide guidelines for constructing more stable regression models in driving studies.

Main Methods:

• Literature review to identify common practices and pitfalls in driving research models.
• Development of a regression model using a sample size of 279 participants, employing best practice techniques.
• Systematic analysis of model performance by randomly reducing sample sizes to assess the participant-to-variable ratio's impact.

Main Results:

• A low ratio of participants to independent variables can lead to over-fitted models and inaccurate conclusions about predictive accuracy.
• Demonstration of how reduced sample sizes exacerbate over-fitting issues in predictive driving models.
• Validation of the importance of adequate sample size for robust model generalizability.

Conclusions:

• Over-fitting is a significant threat to the validity of predictive models in driving research.
• Adhering to specific guidelines, particularly concerning sample size and variable selection, can improve model stability.
• Future driving research should prioritize robust validation techniques and transparent reporting to ensure reliable predictions of driving ability.