进化优化框架用于训练工程应用的多层感知子
Rami Al-Hajj1, Mohamad M Fouad2, Mustafa Zeki1
1College of Engineering and Technology, American University of the Middle East, Kuwait.
Mathematical biosciences and engineering : MBE
|March 8, 2024
概括
这项研究引入了一个进化优化框架来训练神经网络,克服了传统反向传播的局限性,如局部最小值和梯度问题. 新方法显示了竞争性表现和工程数据集的更好的融合.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 优化算法 优化算法
背景情况:
- 训练神经网络的传统监督反向传播算法面临着挑战,包括局部最小停滞和消失/爆炸梯度.
- 这些局限性阻碍了收速度和在损失格局中找到全球最小值的能力.
- 传统方法也需要广泛预选学习参数,影响培训效率.
研究的目的:
- 引入和验证用于训练多层感知子 (MLPs) 的进化优化框架.
- 在神经网络培训中解决传统反向传播算法的局限性.
- 在培训MLP时,对一种新的动态基于小组的合作优化器的性能进行比较.
主要方法:
- 开发了一个进化优化框架,利用动态基于群组的合作优化器.
- 使用拟议的框架训练了多层感知子.
- 验证了五个工程应用数据集的框架.
- 与传统的反向传播和其他进化算法进行性能比较.
主要成果:
- 拟议的进化优化框架在大多数检查的数据集中显示出竞争性表现.
- 与传统方法相比,该框架显示了整体绩效和趋同的改善.
- 对于三个数据集,该框架实现了2.7%,4.83%和5.13%的性能增长,相比于第二好的优化器.
结论:
- 进化优化框架有效地训练了多层感知子,为反向传播提供了可行的替代方案.
- 基于集团的动态合作优化器在克服常见的神经网络训练挑战方面表现有前途.
- 这种方法提高了神经网络的性能和融合,特别是在工程应用中.
相关概念视频
01:22Multi-input and Multi-variable systems
106
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
106
01:29Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
54
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
54
01:25Neural Circuits
1.2K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.2K
01:17Ampere-Maxwell's Law: Problem-Solving
629
A parallel-plate capacitor with capacitance C, whose plates have area A and separation distance d, is connected to a resistor R and a battery of voltage V. The current starts to flow at t = 0. What is the displacement current between the capacitor plates at time t? From the properties of the capacitor, what is the corresponding real current?
To solve the problem, we can use the equations from the analysis of an RC circuit and Maxwell's version of Ampère's law.
For the first part of...
To solve the problem, we can use the equations from the analysis of an RC circuit and Maxwell's version of Ampère's law.
For the first part of...
629
01:20Parallel Processing
151
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
151
01:20Evolutionary Psychology
264
Evolutionary psychology explores the origins of human behavior and mental processes by framing them within the context of natural selection, a theory famously propounded by Charles Darwin. This field asserts that many behaviors common across human societies — ranging from instinctive fear reactions to complex social interactions — arose as evolutionary adaptations. These adaptations enhanced the survival and reproductive success of our ancestors, thereby becoming embedded in the...
264