A universal approximate cross-validation criterion for regular risk functions.

Area of Science:

• Statistical Modeling and Inference
• Machine Learning and Data Science

Background:

• Model selection is crucial in statistical modeling, often involving minimizing an estimating function and assessing with a separate function.
• Classical methods like maximum likelihood estimation (MLE) use cross-entropy for both estimation and assessment (e.g., Akaike Information Criterion - AIC).
• Leave-one-out cross-validation is a common assessment technique but is computationally expensive.

Purpose of the Study:

• To propose a computationally efficient Universal Approximate Cross-Validation criterion under regularity conditions (UACVR).
• To develop a versatile assessment criterion applicable to various estimators and risk functions.
• To provide theoretical underpinnings and empirical validation for the proposed UACVR method.

Main Methods:

• Development of the Universal Approximate Cross-Validation criterion (UACVR) as an approximation to computationally demanding cross-validation.
• Demonstration of UACVR's adaptability to different estimators (penalized likelihood, maximum a posteriori) and risk functions (cross-entropy, CRPS).
• Derivation of asymptotic distributions for UACVR and its differences, enabling statistical inference.

Main Results:

• UACVR offers a computationally feasible alternative to leave-one-out cross-validation.
• The criterion generalizes existing methods, reducing to Takeuchi Information Criterion (TIC) in specific cases.
• Simulations and real-data analysis (psychometrics) confirm the validity and utility of UACVR for comparing estimators.

Conclusions:

• UACVR provides a robust and efficient framework for model selection and estimator assessment.
• The method is broadly applicable across different statistical modeling scenarios.
• UACVR facilitates practical model comparison, especially in complex or data-intensive applications.