Self-Contrastive Forward-Forward algorithm

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Autonomous agents require lifelong learning capabilities compatible with decentralized systems and limited resources.
• Traditional backpropagation for training neural networks requires transpose operations, which are incompatible with purely feedforward approaches.
• The Forward-Forward (FF) algorithm offers a purely feedforward training method but struggles with performance due to unreliable negative data generation in unsupervised learning.

Purpose of the Study:

• To introduce the Self-Contrastive Forward-Forward (SCFF) algorithm, a novel training method designed to enhance the performance of the FF algorithm.
• To address the limitations of existing unsupervised learning methods within the FF framework by improving negative data generation.
• To expand the applicability of FF-based algorithms to sequential data tasks by integrating them with recurrent neural networks.

Main Methods:

• The proposed Self-Contrastive Forward-Forward (SCFF) algorithm is inspired by self-supervised contrastive learning techniques.
• SCFF generates both positive and negative data samples suitable for diverse datasets, overcoming limitations of previous FF negative data generation.
• The study extends the FF algorithm's application to recurrent neural networks for processing sequential data.

Main Results:

• SCFF demonstrates superior performance compared to existing unsupervised local learning algorithms on multiple benchmark datasets (MNIST, CIFAR-10, STL-10, Tiny ImageNet).
• The algorithm successfully extends the FF approach to training recurrent neural networks, proving effective for sequential data.
• The research validates SCFF as a competitive training method for unsupervised local learning.

Conclusions:

• The SCFF algorithm significantly closes the performance gap for the Forward-Forward algorithm in unsupervised learning tasks.
• This work enables high-accuracy, real-time learning for resource-constrained edge devices by improving autonomous system training.
• The successful application to recurrent neural networks broadens the utility of FF-based methods for diverse machine learning applications.