Training convolutional neural networks with the Forward-Forward Algorithm.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Computational Neuroscience

Background:

• Deep neural networks, particularly Convolutional Neural Networks (CNNs), dominate image analysis, primarily trained with backpropagation (BP).
• Geoffrey Hinton proposed the Forward-Forward (FF) algorithm as a biologically plausible alternative, utilizing local goodness functions and joint presentation of positive/negative data.
• Extending FF to CNNs requires novel methods for label information propagation across spatial locations.

Purpose of the Study:

• To adapt the Forward-Forward (FF) algorithm for use with Convolutional Neural Networks (CNNs).
• To develop and evaluate spatially extended labeling strategies for FF-trained CNNs.
• To investigate the learning dynamics, stability, and feature learning capabilities of FF-trained CNNs.

Main Methods:

• Introduced two spatially extended labeling strategies: Fourier patterns and morphological transformations.
• Applied these strategies to train deeper CNNs on CIFAR10 and CIFAR100 datasets using the FF algorithm.
• Utilized Class Activation Maps (CAMs) to analyze learned features.

Main Results:

• Demonstrated successful optimization of deeper FF-trained CNNs on image datasets.
• Showcased that morphology-based labels effectively prevent shortcut solutions on complex datasets.
• Confirmed that FF training scales effectively to 100 classes (CIFAR100) and that FF-CNNs learn meaningful, complementary features across layers.

Conclusions:

• FF training is feasible and effective for Convolutional Neural Networks, extending beyond fully connected architectures.
• Spatially extended labeling strategies enable FF CNNs to learn effectively and avoid shortcut solutions.
• FF-trained CNNs exhibit biologically plausible learning dynamics and hold potential for neuromorphic computing.