自我对比的前进前进算法
Xing Chen1, Dongshu Liu2, Jérémie Laydevant3,4
1Laboratoire Albert Fert, CNRS, Thales, Université Paris-Saclay, Palaiseau, France. xing.chen@cnrs-thales.fr.
Nature communications
|July 2, 2025
概括
自对比前进 (SCFF) 算法通过增强无监督学习来改善自主系统训练. SCFF在基准数据集上实现了竞争性表现,使边缘设备上的实时学习成为可能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 自主代理人需要与分散的系统和有限的资源兼容的终身学习能力.
- 训练神经网络的传统反向传播需要转换操作,这与纯粹的前方法不相容.
- 前进前进 (FF) 算法提供了一个纯粹的前培训方法,但由于在无监督学习中不可靠的负数据生成,它在性能方面陷入困境.
研究的目的:
- 引入自我对比的前进前进 (SCFF) 算法,这是一种旨在提高FF算法的性能的新型训练方法.
- 通过改善负数据生成,解决FF框架内现有的无监督学习方法的局限性.
- 通过将它们与循环神经网络集成,扩大基于FF的算法的适用于顺序数据任务.
主要方法:
- 拟议的自我对比前进 (SCFF) 算法受到自我监督对比学习技术的启发.
- SCFF生成了适合不同数据集的正和负数据样本,克服了以前的FF负数据生成的局限性.
- 该研究将FF算法的应用扩展到循环神经网络,用于处理序列数据.
主要成果:
- 与现有的无监督本地学习算法相比,SCFF在多个基准数据集 (MNIST,CIFAR-10,STL-10,Tiny ImageNet) 上表现出卓越的性能.
- 该算法成功地将FF方法扩展到训练循环神经网络,并证明对顺序数据有效.
- 该研究验证了SCFF作为无监督本地学习的竞争性培训方法.
结论:
- 在无监督学习任务中,SCFF算法显著缩小了Forward-Forward算法的性能差距.
- 这项工作通过改进自主系统培训,使资源有限的边缘设备能够实现高精度的实时学习.
- 对循环神经网络的成功应用扩大了基于FF的方法对各种机器学习应用的实用性.
相关概念视频
03:00Self-Evaluation: Self-Enhancement and Self-Verification
5.4K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
5.4K
02:57The Sense of Self: Reflected Self-Appraisal and Social Comparison
51.8K
According to Charles Cooley, we base our image on what we think other people see (Cooley 1902). We imagine how we must appear to others, then react to this speculation. We don certain clothes, prepare our hair in a particular manner, wear makeup, use cologne, and the like—all with the notion that our presentation of ourselves is going to affect how others perceive us. We expect a certain reaction, and, if lucky, we get the one we desire and feel good about it. But more than that, Cooley...
51.8K
01:25The Anchoring-and-Adjustment Heuristic
7.4K
In order to make good decisions, we use our knowledge and our reasoning. Often, this knowledge and reasoning is sound and solid. However, sometimes, we are swayed by biases or by others manipulating a situation. For example, let’s say you and three friends wanted to rent a house and had a combined target budget of $1,600. The realtor shows you only very run-down houses for $1,600 and then shows you a very nice house for $2,000. Might you ask each person to pay more in rent to get the...
7.4K
02:45Self-Discrepancy Theory
18.5K
One influential perspective on what motivates people's behavior is detailed in Tory Higgin's self-discrepancy theory (Higgins, 1987). He proposed that people hold disagreeing internal representations of themselves that lead to different emotional states.
18.5K
01:17Sign Test for Matched Pairs
214
The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in...
To conduct the sign test, we first calculate the differences in...
214
01:12Hindsight Biases
3.9K
Hindsight bias leads you to believe that the event you just experienced was predictable, even though it really wasn’t. In other words, you knew all along that things would turn out the way they did. Can you relate this to the phrase "Hindsight is 20/20" now?
3.9K