VFF-Net: Evolving forward-forward algorithms into convolutional neural networks for enhanced computational insights

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Traditional back-propagation (BP) algorithms face limitations like overfitting, vanishing gradients, and slow convergence.
• The forward-forward network (FFN) emerged as an alternative but requires further improvements for deep network applications.

Purpose of the Study:

• To introduce a Visual Forward-Forward Network (VFF-Net) that enhances FFN performance, particularly for deep convolutional neural network (CNN) training.
• To address input information loss and performance degradation issues in FFNs when applied to CNNs.

Main Methods:

• VFF-Net employs label-wise noise labeling and cosine-similarity-based contrastive loss using intermediate features.
• Layer grouping is utilized to optimize multiple minima and facilitate transfer learning in existing CNN models.
• Ensemble training effects are demonstrated to enhance model generalization.

Main Results:

• VFF-Net reduced test error by up to 8.31% on CIFAR-10 and 3.80% on CIFAR-100 compared to standard FFNs in CNNs.
• A fully connected layer-based VFF-Net achieved 1.70% test error on MNIST, outperforming existing BP methods.
• The proposed VFF-Net significantly narrows the performance gap between FFNs and BP.

Conclusions:

• VFF-Net offers a substantial improvement over standard FFNs, making them more competitive with BP algorithms.
• The VFF-Net architecture demonstrates flexibility and portability to existing CNN-based models, indicating broad applicability.