Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Improved learning algorithms for mixture of experts in multiclass classification.

K Chen1, L Xu, H Chi

  • 1Department of Computer Science and Engineering, Chinese University of Hong Kong, Shatin, NT, Hong Kong, People's Republic of China

Neural Networks : the Official Journal of the International Neural Network Society
|March 29, 2003
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

The small-cage induced sedentariness in male young rats: evidence from energy expenditure and glucose uptake.

Scientific reports·2026
Same author

Short bouts and long-term exercise reduce sedentary-induced bone loss and microstructural changes by modulating bone formation and resorption in healthy young male rats.

Scientific reports·2025
Same author

Association between sedentary behavior and bone mass, microstructure and strength in children, adolescents and young adults: a systematic review.

BMC public health·2024
Same author

[A transcriptomic analysis of correlation between mitochondrial function and energy metabolism remodeling in mice with myocardial fibrosis following myocardial infarction].

Nan fang yi ke da xue xue bao = Journal of Southern Medical University·2024
Same author

[Learning curve and clinical efficacy of aortic surgery through upper hemisternotomy].

Nan fang yi ke da xue xue bao = Journal of Southern Medical University·2023
Same author

The dUTPase of caprine arthritis-encephalitis virus negatively regulates interferon signaling pathway.

Iranian journal of veterinary research·2021
Same journal

Anchor-based disentanglement framework for incremental multi-view clustering.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Complex-valued amplitude-phase interference modeling for adversarially robust classification.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

TraNce: Type-aware hypergraph neural network with biological mediators for drug repositioning.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Decentralized ADMM for factorization-based Low-rank matrix estimation.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Memristive neuromorphic circuit design inspired by the neural mechanisms of conditioned fear.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Q-learning based asynchronous Boolean control networks stabilization with data loss.

Neural networks : the official journal of the International Neural Network Society·2026
See all related articles

This study addresses instability in Mixture of Experts (ME) models trained with Expectation-Maximization (EM) and Iteratively Reweighted Least Squares (IRLS) for multiclass classification. Researchers propose Newton-Raphson methods, including an approximation, to improve ME model performance and stability.

Area of Science:

  • Machine Learning
  • Artificial Intelligence
  • Computational Neuroscience

Background:

  • Mixture of Experts (ME) is a modular neural network architecture for supervised learning.
  • The Expectation-Maximization (EM) algorithm, utilizing Iteratively Reweighted Least Squares (IRLS) in its inner loop, is commonly used for ME parameter adjustment.
  • Instability and poor performance of ME models in multiclass classification have been reported when trained with the standard EM-IRLS approach.

Purpose of the Study:

  • To investigate the cause of instability in the EM-IRLS algorithm for ME models in multiclass classification.
  • To develop improved learning algorithms for ME architectures that enhance stability and performance.
  • To propose and evaluate a computationally efficient approximation to the Newton-Raphson method for ME training.

Related Experiment Videos

Main Methods:

  • Explored the instability of the IRLS algorithm within the EM framework, identifying an incorrect assumption leading to an incomplete Hessian matrix.
  • Applied the Newton-Raphson method with an exact Hessian matrix to the inner loop of the EM algorithm for multiclass classification.
  • Developed and validated an approximation to the Newton-Raphson algorithm using a generalized Bernoulli density to mitigate computational costs.

Main Results:

  • The proposed Newton-Raphson methods successfully resolve the instability issue associated with the EM-IRLS algorithm in ME models.
  • Both the exact and approximated Newton-Raphson algorithms demonstrate significantly improved performance in multiclass classification tasks.
  • The approximation algorithm achieves fast learning speeds and its limitations are empirically analyzed.

Conclusions:

  • The identified incorrect assumption in the IRLS algorithm is the source of instability in EM-trained ME models for multiclass problems.
  • Newton-Raphson-based learning algorithms offer a robust and effective solution for training ME architectures, leading to superior classification performance.
  • The proposed approximation algorithm provides an efficient alternative for practical applications requiring fast and stable ME model training.