Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Bayesian model assessment and comparison using cross-validation predictive densities.

Aki Vehtari1, Jouko Lampinen

  • 1Laboratory of Computational Engineering, Helsinki University of Technology, FIN-02015, HUT, Finland. Aki.Vehtari@hut.fi

Neural Computation
|October 25, 2002
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Pathfinder: Parallel quasi-Newton variational inference.

Journal of machine learning research : JMLR·2026
Same author

Raw signal segmentation for estimating RNA modification from Nanopore direct RNA sequencing data.

eLife·2026
Same author

A Framework for Improving the Reliability of Black-box Variational Inference.

Journal of machine learning research : JMLR·2025
Same author

Simulation-Based Calibration Checking for Bayesian Computation: The Choice of Test Quantities Shapes Sensitivity.

Bayesian analysis·2025
Same author

Bayesian Hierarchical Stacking: Some Models Are (Somewhere) Useful.

Bayesian analysis·2025
Same author

Fast Methods for Posterior Inference of Two-Group Normal-Normal Models.

Bayesian analysis·2025
Same journal

A Model-Free Reinforcement Learning Implementation of Decision Making Under Uncertainty by Sequential Sampling.

Neural computation·2026
Same journal

DROP: Distributional and Regular Optimism and Pessimism for Reinforcement Learning.

Neural computation·2026
Same journal

Hierarchical Active Inference Using Successor Representations.

Neural computation·2026
Same journal

W-Kernel and Its Principal Space for Frequentist Evaluation of Bayesian Estimators.

Neural computation·2026
Same journal

A Hidden Markov Model-Inspired Sequence Classification Method for Hyperdimensional Computing.

Neural computation·2026
Same journal

Sparse Graphical Modeling for Electrophysiological Phase-Based Connectivity Using Circular Statistics.

Neural computation·2026
See all related articles

This study introduces methods for comparing complex Bayesian models by estimating their future predictive performance using expected utilities. It details how cross-validation and Bayesian bootstrap help assess model uncertainty and facilitate model selection.

Area of Science:

  • Statistics
  • Machine Learning
  • Computational Science

Background:

  • Assessing complex hierarchical Bayesian models is challenging.
  • Estimating future predictive capability is crucial for model evaluation.
  • Understanding uncertainty in model estimates is vital for reliable comparisons.

Purpose of the Study:

  • To present practical methods for assessing, comparing, and selecting complex hierarchical Bayesian models.
  • To introduce a framework for estimating and quantifying uncertainty in expected utility estimates.
  • To enable robust model comparison through probabilistic assessments.

Main Methods:

  • Utilizing cross-validation predictive densities for expected utility estimation.
  • Employing Bayesian bootstrap for sampling from the distribution of expected utility estimates.

Related Experiment Videos

  • Analyzing the properties of importance sampling and k-fold cross-validation techniques.

    • Main Results:

    • Demonstrated the utility of expected utility distributions for model comparison.
    • Showcased the effectiveness of the proposed cross-validation and Bayesian bootstrap approach.
    • Successfully applied the methods to multilayer perceptron neural networks and Gaussian processes.

    Conclusions:

    • The proposed methods provide a robust framework for evaluating and selecting complex Bayesian models.
    • Quantifying uncertainty in predictive performance is essential for reliable model assessment.
    • The approach is applicable to various machine learning models and real-world problems.