Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Bayesian model assessment and comparison using cross-validation predictive densities.

Aki Vehtari1, Jouko Lampinen

  • 1Laboratory of Computational Engineering, Helsinki University of Technology, FIN-02015, HUT, Finland. Aki.Vehtari@hut.fi

Neural Computation
|October 25, 2002
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Pathfinder: Parallel quasi-Newton variational inference.

Journal of machine learning research : JMLR·2026
Same author

Raw signal segmentation for estimating RNA modification from Nanopore direct RNA sequencing data.

eLife·2026
Same author

A Framework for Improving the Reliability of Black-box Variational Inference.

Journal of machine learning research : JMLR·2025
Same author

Simulation-Based Calibration Checking for Bayesian Computation: The Choice of Test Quantities Shapes Sensitivity.

Bayesian analysis·2025
Same author

Bayesian Hierarchical Stacking: Some Models Are (Somewhere) Useful.

Bayesian analysis·2025
Same author

Fast Methods for Posterior Inference of Two-Group Normal-Normal Models.

Bayesian analysis·2025

This study introduces methods for comparing complex Bayesian models by estimating their future predictive performance using expected utilities. It details how cross-validation and Bayesian bootstrap help assess model uncertainty and facilitate model selection.

Area of Science:

  • Statistics
  • Machine Learning
  • Computational Science

Background:

  • Assessing complex hierarchical Bayesian models is challenging.
  • Estimating future predictive capability is crucial for model evaluation.
  • Understanding uncertainty in model estimates is vital for reliable comparisons.

Purpose of the Study:

  • To present practical methods for assessing, comparing, and selecting complex hierarchical Bayesian models.
  • To introduce a framework for estimating and quantifying uncertainty in expected utility estimates.
  • To enable robust model comparison through probabilistic assessments.

Main Methods:

  • Utilizing cross-validation predictive densities for expected utility estimation.
  • Employing Bayesian bootstrap for sampling from the distribution of expected utility estimates.

Related Experiment Videos

  • Analyzing the properties of importance sampling and k-fold cross-validation techniques.

    • Main Results:

    • Demonstrated the utility of expected utility distributions for model comparison.
    • Showcased the effectiveness of the proposed cross-validation and Bayesian bootstrap approach.
    • Successfully applied the methods to multilayer perceptron neural networks and Gaussian processes.

    Conclusions:

    • The proposed methods provide a robust framework for evaluating and selecting complex Bayesian models.
    • Quantifying uncertainty in predictive performance is essential for reliable model assessment.
    • The approach is applicable to various machine learning models and real-world problems.