Model Reduction Through Progressive Latent Space Pruning in Deep Active Inference.

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Machine learning

Background:

• Sleep is crucial for learning and synaptic pruning, analogous to parameter learning and Bayesian model reduction in active inference.
• Deep active inference models require pre-specified latent space dimensionality, posing a challenge for model optimization.

Purpose of the Study:

• To investigate methods for reducing the dimensionality of latent spaces in deep active inference models.
• To explore how these reductions impact model complexity, performance, and generalization.

Main Methods:

• Two novel methods were developed: one for post-hoc model reduction (sleep-like) and another for during-training reduction (reflection-like) with "aha" moments.
• Deep neural networks were used for state space construction within the active inference framework.

Main Results:

• The sleep-like method retained model performance, while the reflection-like method showed only a slight decrease.
• Real-world data reconstructions were indistinguishable before and after latent space reduction.

Conclusions:

• Latent space pruning in deep active inference is feasible and beneficial for model complexity reduction.
• A trade-off exists between training time and model accuracy/generalization, influenced by the chosen reduction method.