Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representations.

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Machine learning

Background:

• Deep-predictive-coding networks (DPCNs) are hierarchical generative models utilizing feed-forward and feedback connections for dynamic, context-sensitive feature representation.
• A key component, the forward-backward inference, is computationally intensive and causes learning stagnation, limiting network depth.

Purpose of the Study:

• To identify the cause of the computational bottleneck in DPCN inference.
• To propose a novel, accelerated forward-inference strategy to overcome learning stagnation.
• To demonstrate the construction of deeper and wider predictive-coding networks with enhanced feature representation capabilities.

Main Methods:

• Theoretical analysis to pinpoint the cause of the inference bottleneck in DPCNs.
• Development of a new forward-inference strategy employing accelerated proximal gradients.
• Implementation and evaluation of deep and wide convolutional DPCNs utilizing the new strategy.

Main Results:

• The study elucidates the reasons behind the computational bottleneck in DPCN inference.
• The proposed accelerated proximal gradient strategy demonstrates faster convergence and overcomes learning stagnation.
• Constructed convolutional DPCNs exhibit improved receptive fields for comprehensive object class capture.
• Unsupervised object recognition performance surpasses convolutional autoencoders and matches supervised convolutional networks.

Conclusions:

• The accelerated forward-inference strategy effectively resolves the computational bottleneck in DPCNs.
• This advancement enables the creation of significantly deeper and wider predictive-coding architectures.
• The enhanced DPCNs achieve superior unsupervised object recognition, rivaling supervised methods.