Effect of Top-Down Connections in Hierarchical Sparse Coding.

Area of Science:

• Computational neuroscience
• Machine learning
• Image processing

Background:

• Hierarchical sparse coding (HSC) efficiently represents complex data like images.
• Traditional HSC models often use independent layers, which may not fully capture data structure.
• Predictive coding (PC) theory suggests incorporating top-down connections for improved neural processing.

Purpose of the Study:

• To introduce and evaluate a novel 2-layer sparse predictive coding (2L-SPC) model.
• To assess the impact of interlayer feedback connections on HSC performance.
• To compare 2L-SPC against a hierarchical Lasso (Hi-La) network with independent layers.

Main Methods:

• Developed a 2L-SPC model incorporating top-down feedback connections.
• Trained and compared 2L-SPC and a 2-layer Hi-La network on four diverse datasets.
• Varied sparsity parameters across layers during training.

Main Results:

• 2L-SPC achieved lower overall prediction error due to efficient error transfer via feedback.
• The inference stage in 2L-SPC converged faster and produced more refined representations.
• Top-down connections in 2L-SPC accelerated the learning process for HSC.
• Emergent features in 2L-SPC were more generic with larger spatial extensions.

Conclusions:

• Interlayer feedback connections significantly enhance hierarchical sparse coding performance.
• The 2L-SPC model offers advantages in prediction accuracy, inference speed, and feature representation.
• This work supports the integration of predictive coding principles into computational models for structured data representation.