Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Survival Tree01:19

Survival Tree

73
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
 Building a Survival Tree
Constructing a...
73
Weighted Mean00:57

Weighted Mean

5.0K
While taking the arithmetic, geometric, or harmonic mean of a sample data set, equal importance is assigned to all the data points. However, all the values may not always be equally important in some data sets. An intrinsic bias might make it more important to give more weightage to specific values over others.
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
5.0K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

45
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
45
Entropy Change in Reversible Processes01:10

Entropy Change in Reversible Processes

2.5K
In the Carnot engine, which achieves the maximum efficiency between two reservoirs of fixed temperatures, the total change in entropy is zero. The observation can be generalized by considering any reversible cyclic process consisting of many Carnot cycles. Thus, it can be stated that the total entropy change of any ideal reversible cycle is zero.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
2.5K
Entropy and the Second Law of Thermodynamics01:20

Entropy and the Second Law of Thermodynamics

2.8K
The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation  between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
2.8K
Regression Toward the Mean01:52

Regression Toward the Mean

6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Effect of Inorganic Fillers on Electrical and Mechanical Properties of Ceramizable Silicone Rubber.

Polymers·2024
Same author

Positron emission tomography with Pittsburgh compound B in diagnosis of early stage Alzheimer's disease.

Cell biochemistry and biophysics·2010
Same author

[Preparation of enteric nanoparticles of Schisandra total lignanoids and preliminary study on its pharmacokinetics].

Yao xue xue bao = Acta pharmaceutica Sinica·2010
Same author

[A survey of health effects on population exposure to a dust event in Beijing City].

Wei sheng yan jiu = Journal of hygiene research·2010
Same author

Effects of beta-ionone on mammary carcinogenesis and antioxidant status in rats treated with DMBA.

Nutrition and cancer·2010
Same author

[The effect of urokinase on hepatic fibrogenesis in rats].

Zhonghua gan zang bing za zhi = Zhonghua ganzangbing zazhi = Chinese journal of hepatology·2009

Related Experiment Video

Updated: Jun 14, 2025

Using Wavelet Entropy to Demonstrate how Mindfulness Practice Increases Coordination between Irregular Cerebral and Cardiac Activities
08:08

Using Wavelet Entropy to Demonstrate how Mindfulness Practice Increases Coordination between Irregular Cerebral and Cardiac Activities

Published on: May 10, 2017

14.7K

WI-TMLEGA: Weight Initialization and Training Method Based on Entropy Gain and Learning Rate Adjustment.

Hongchuan Tang1, Zhongguo Li1, Qi Wang1,2

  • 1School of Mechanical Engineering, Jiangsu University of Science and Technology, Zhenjiang 212100, China.

Entropy (Basel, Switzerland)
|August 29, 2024
PubMed
Summary

This study introduces an entropy gain method for weight initialization and dynamic learning rates in multilayer perceptron (MLP) models. The approach significantly enhances training effectiveness and recognition accuracy for large models.

Keywords:
MNIST datasetlearning ratemultilayer perceptronweight initializationweight update

More Related Videos

Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.0K
Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
11:15

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy

Published on: June 27, 2013

33.7K

Related Experiment Videos

Last Updated: Jun 14, 2025

Using Wavelet Entropy to Demonstrate how Mindfulness Practice Increases Coordination between Irregular Cerebral and Cardiac Activities
08:08

Using Wavelet Entropy to Demonstrate how Mindfulness Practice Increases Coordination between Irregular Cerebral and Cardiac Activities

Published on: May 10, 2017

14.7K
Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.0K
Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
11:15

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy

Published on: June 27, 2013

33.7K

Area of Science:

  • Machine Learning
  • Artificial Intelligence
  • Deep Learning

Background:

  • Prolonged training times and low recognition rates are significant challenges in large model applications.
  • Current weight initialization and learning rate adjustment methods may not be optimal for complex models.
  • Multilayer Perceptron (MLP) models serve as a foundational example for evaluating new training techniques.

Purpose of the Study:

  • To propose and evaluate a novel weight training method for large models.
  • To address issues of prolonged training times and low recognition rates.
  • To demonstrate the efficacy of entropy gain for weight initialization and dynamic learning rate adjustment.

Main Methods:

  • Utilized entropy gain to replace random initial values for weight initialization.
  • Implemented an incremental learning rate strategy for dynamic weight updates.
  • Trained and validated the multilayer perceptron (MLP) model using the MNIST handwritten digit dataset.

Main Results:

  • The proposed initialization method improved training effectiveness by 39.8% compared to random initialization.
  • Achieved an increase in maximum recognition accuracy by 8.9%.
  • Demonstrated significant performance gains on the MNIST dataset.

Conclusions:

  • The proposed entropy gain-based weight training method is feasible and effective for large model applications.
  • The method offers a viable solution to enhance both training efficiency and recognition accuracy.
  • Further research can explore this technique in other complex deep learning architectures.