Related Experiment Video
Updated: Aug 19, 2025
13:19
Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
9.3K
A fractional gradient descent algorithm robust to the initial weights of multilayer perceptron
Xuetao Xie1, Yi-Fei Pu1, Jian Wang2
1College of Computer Science, Sichuan University, Chengdu, 610065, China.
Summary
This study introduces a robust fractional gradient descent (RFGD) algorithm for multilayer perceptrons (MLP). RFGD demonstrates superior performance and robustness against initial weight variations compared to other optimization methods.
Area of Science:
- Artificial Intelligence
- Machine Learning
- Optimization Algorithms
Background:
- Initial weights critically impact multilayer perceptron (MLP) performance.
- Existing optimization algorithms may exhibit sensitivity to weight initialization.
Purpose of the Study:
- Propose a novel fractional gradient descent (RFGD) algorithm.
- Enhance MLP robustness against varying initial weights.
- Analyze the effectiveness and convergence of RFGD.
Main Methods:
- Developed RFGD based on enhanced fractional derivatives and convex optimization.
- Analyzed algorithm convergence and computational complexity.
- Conducted numerical experiments comparing RFGD with GD, Adam, Padam, AdaBelief, and AdaDiff.
Main Results:
- RFGD exhibits strong robustness to the fractional calculus order.
- RFGD demonstrates superior robust performance for MLP initial weights compared to GD, Adam, Padam, AdaBelief, and AdaDiff.
- Theoretical analysis of RFGD correctness was verified.
Conclusions:
- The proposed RFGD algorithm offers significant improvements in MLP performance robustness.
- RFGD provides a viable alternative optimization method, particularly when initial weight sensitivity is a concern.
Keywords:
ConvergenceFirst derivativeFractional calculusInitial weightsMultilayer perceptron (MLP)RobustMore Related Videos
Related Concept Videos
01:29Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
97
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
97
01:11Residuals and Least-Squares Property
7.8K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.8K
01:52Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
01:26Linear Approximation in Frequency Domain
123
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
123
01:18Reducing Line Loss
186
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
186
01:07Time-Domain Interpretation of PD Control
165
Proportional-Derivative (PD) control is a widely used control method in various engineering systems to enhance stability and performance. In a system with only proportional control, common issues include high maximum overshoot and oscillation, observed in both the error signal and its rate of change. This behavior can be divided into three distinct phases: initial overshoot, subsequent undershoot, and gradual stabilization.
Consider the example of control of motor torque. Initially, a positive...
Consider the example of control of motor torque. Initially, a positive...
165