Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Objective functions of online weight noise injection training algorithms for MLPs.

Kevin Ho1, Chi-Sing Leung, John Sum

  • 1Department of Computer Science and Communication Engineering, Providence University, Taichung 43301, Taiwan. ho@pu.edu.tw

IEEE Transactions on Neural Networks
|December 30, 2010
PubMed
Summary

Weight noise injection in multilayer perceptrons (MLPs) improves fault tolerance. This study clarifies misconceptions, showing additive noise aligns with prediction error, while multiplicative noise differs, resolving training objective misunderstandings.

Related Concept Videos

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
Methods of Medium Optimization01:28

Methods of Medium Optimization

Optimizing growth media enhances microbial proliferation and maximizes product yield. Statistical experimental design methodologies provide structured and reproducible approaches, offering progressively higher levels of robustness and efficiency.The One-Factor-at-a-Time (OFAT) MethodThe One-Factor-at-a-Time (OFAT) method involves adjusting a single variable while keeping all others constant. However, it cannot detect interactions between variables, often leading to suboptimal outcomes when...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Embolic Agent Selection and Outcomes in Renal Angiomyolipoma Embolisation.

Journal of medical imaging and radiation oncology·2026
Same author

Machine Learning-Driven Prediction of Intensive Care Units Mortality and Length of Stay: A 11-Year Retrospective Study in Hong Kong Public Hospitals.

Journal of medical systems·2026
Same author

Comparing infectious risk of Trastuzumab-deruxtecan to Trastuzumab-emtansine in patients with breast cancer.

Breast cancer research and treatment·2026
Same author

A High-Accuracy Probabilistic-Based Sigmoid Approximator Incorporating Memory-Saving and Time-Efficient Strategies.

IEEE transactions on neural networks and learning systems·2026
Same author

Marginal Band Collapsing: Structural Insights Into Shear-Induced Platelet Activation Relevant to Circulatory Assist Devices.

ASAIO journal (American Society for Artificial Internal Organs : 1992)·2026
Same author

Health-economic evaluation of a novel personalized digital avatar-based anemia management software in hemodialysis patients.

NPJ digital medicine·2026

Area of Science:

  • Machine Learning
  • Artificial Intelligence
  • Neural Networks

Background:

  • Weight noise injection is a long-standing method to enhance the fault tolerance of multilayer perceptrons (MLPs).
  • Existing training algorithms for MLPs with weight noise may be based on misunderstood objective functions.
  • Misconceptions exist regarding the equivalence between prediction error and the objective function in weight noise injection algorithms.

Purpose of the Study:

  • To clarify misconceptions surrounding the objective functions minimized by weight noise injection algorithms in MLPs.
  • To differentiate the objective functions for additive versus multiplicative weight noise injection.
  • To provide accurate analytical insights into MLP training dynamics with weight noise.

Main Methods:

Related Experiment Videos

  • Analysis of mean updating equations for two weight noise injection scenarios: additive and multiplicative.
  • Mathematical derivation to determine the true objective functions for each noise injection type.
  • Comparison of derived objective functions with prediction errors of faulty MLPs.

    • Main Results:

    • For additive weight noise injection, the objective function is equivalent to the prediction error of a faulty MLP, comprising mean square error and a smoothing regularizer.
    • For multiplicative weight noise injection, the objective function deviates from the prediction error of a faulty MLP.
    • The study resolves existing discrepancies in understanding MLP training with weight noise.

    Conclusions:

    • The precise objective functions for additive and multiplicative weight noise injection in MLPs have been identified.
    • Clarification of these objectives corrects prevalent misconceptions in the field.
    • Accurate understanding facilitates more effective training strategies for robust MLPs.