Predictive Coding for Dynamic Visual Processing: Development of Functional Hierarchy in a Multiple Spatiotemporal...

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Robotics

Background:

• Predictive coding models are increasingly used to understand neural processing.
• Modeling human movement requires capturing complex spatiotemporal dynamics.
• Recurrent neural networks offer a framework for sequential data processing.

Purpose of the Study:

• To propose a novel neural network architecture, the predictive multiple spatiotemporal scales recurrent neural network (P-MSTRNN).
• To investigate the P-MSTRNN's ability to predict and imitate human whole-body cyclic movement patterns.
• To analyze the role of learning stages and network dynamics in task performance.

Main Methods:

• Developed a predictive coding type neural network (P-MSTRNN) with multiscale spatiotemporal constraints.
• Utilized differently sized receptive fields and time constants across network layers.
• Employed regression of prediction error for inferring movement intentions and imitation.
• Examined model performance during pattern generation and predictive imitation across learning stages.

Main Results:

• The P-MSTRNN successfully learned to predict human whole-body cyclic movement patterns.
• The network developed a functional hierarchy with distinct dynamic structures at each layer.
• The number of limit cycle attractors increased with learning, corresponding to target movement patterns.
• Transient dynamics early in learning facilitated successful pattern generation and imitation.

Conclusions:

• The P-MSTRNN effectively models human movement prediction and imitation.
• Exploiting transient dynamics during early learning is crucial for successful task performance.
• The model's hierarchical structure and multiscale processing contribute to its capabilities.