Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Bayesian model search for mixture models based on optimizing variational bounds.

Naonori Ueda1, Zoubin Ghahramani

  • 1NTT Communication Science Laboratories, Soraku-gun, Kyoto, Japan. ueda@cslab.kecl.ntt.co.jp

Neural Networks : the Official Journal of the International Neural Network Society
|November 12, 2002
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Nonlinear combinatorial analysis of blood transcriptomes identifies PRKAR1A as a regulator of TDP-43 pathophysiology in amyotrophic lateral sclerosis.

Biology methods & protocols·2026
Same author

Subset binding enables detection of multimodal patient subgroup patterns and drug target discovery in idiopathic pulmonary fibrosis.

Briefings in bioinformatics·2026
Same author

Clinically informed intermediate reasoning enables generalizable prostate cancer prognostication through machine learning in limited settings.

NPJ digital medicine·2025
Same author

Multi-horizon prediction of tropical cyclone intensity and its interpretability with temporal fusion transformer.

Scientific reports·2025
Same author

Current status and future direction of cancer research using artificial intelligence for clinical application.

Cancer science·2024
Same author

A novel deep learning approach for typhoon-induced storm surge modeling through efficient emulation of wind and pressure fields.

Scientific reports·2023
Same journal

Exploiting audio-visual modalities in videos: Object detection via multi-stage bilateral coupling network.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Reliability-aware modality completion with cross-modal distillation for federated learning with missing modalities.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

IGFD-Net: Illumination-guided frequency decoupling for polarization image fusion.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Multiple-Strategies dung beetle optimizer and its applications in engineering optimization and bankruptcy prediction.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Aggregating global-scale pixel-wise forgery cues within a graph.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Finite-Time intermittent control for secure synchronization of Neutral-Type stochastic delayed neural networks under aperiodic DoS attacks.

Neural networks : the official journal of the International Neural Network Society·2026
See all related articles

This study introduces a variational Bayesian (VB) method to overcome local optima and model structure issues in mixture models. The approach effectively determines the optimal number of components, such as in Mixture of Experts (MoE), avoiding common learning pitfalls.

Area of Science:

  • Machine Learning
  • Statistical Modeling
  • Artificial Intelligence

Background:

  • Learning mixture models is challenged by local optima and determining the correct model structure.
  • Existing methods often struggle to address both parameter optimization and structure determination simultaneously.

Purpose of the Study:

  • To present a unified variational Bayesian (VB) framework for simultaneously optimizing mixture model parameters and structure.
  • To address the persistent problems of local optima and model structure determination in mixture modeling.

Main Methods:

  • Derivation of an objective function within the VB framework to optimize both model parameter distributions and structure.
  • Development of a deterministic algorithm using split and merge operations for approximate optimization.

Related Experiment Videos

  • Application to Mixture of Experts (MoE) models.

    • Main Results:

    • The proposed VB method successfully optimizes model structure and parameters concurrently.
    • The deterministic algorithm effectively approximates the objective function using split and merge operations.
    • Experimental results on MoE models demonstrate the ability to find the optimal number of experts.

    Conclusions:

    • The presented VB framework offers a robust solution for learning mixture models, overcoming local optima and structure determination challenges.
    • The method is effective in identifying the optimal model complexity, as shown with MoE models.
    • This approach advances the field of mixture model learning by providing a simultaneous optimization strategy.