Deep predictive coding with bi-directional propagation for classification and reconstruction

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Predictive Coding (PC) is a brain information processing theory where layers predict preceding layer activities for local error computation and parallel learning.
• Existing PC methods offer foundational learning principles but can be enhanced for simultaneous task performance.

Purpose of the Study:

• To introduce Deep Bi-directional Predictive Coding (DBPC) as a novel learning algorithm for neural networks.
• To enable simultaneous classification and reconstruction tasks using a single set of learned weights.
• To enhance learning efficiency through local information utilization and in-parallel training across network layers.

Main Methods:

• DBPC trains networks by having each layer predict activities of both previous and next layers, facilitating feedforward and feedback propagation.
• The algorithm supports training of both fully connected and convolutional neural networks.
• Learning relies on locally available information, enabling parallel computation across all network layers.

Main Results:

• DBPC achieved high classification accuracies on MNIST (99.58%), Fashion-MNIST (92.42%), and CIFAR-10 (74.29%), surpassing established PC benchmarks.
• Performance is competitive with state-of-the-art Error-Backpropagation methods on several datasets.
• DBPC utilizes significantly smaller networks compared to benchmarks while enabling input reconstruction from all learned representations.

Conclusions:

• DBPC offers an efficient training protocol by leveraging local information and in-parallel learning mechanisms.
• The algorithm effectively performs both classification and reconstruction tasks simultaneously.
• DBPC presents a more efficient approach for training versatile neural networks.