Supervised structure learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Structure learning is crucial for understanding discrete generative models.
• Bayesian model selection offers a principled framework for learning.
• The order of data assimilation can significantly impact model discovery.

Purpose of the Study:

• To develop a Bayesian method for structure learning in discrete generative models.
• To investigate the role of data ingestion order in model selection.
• To utilize expected free energy for guiding model discovery.

Main Methods:

• Employing Bayesian model selection with priors on model selection based on expected free energy.
• Reformulating expected free energy as constrained mutual information.
• Applying the scheme to image classification (MNIST) and dynamic model discovery (sprite-based disentanglement, Tower of Hanoi).

Main Results:

• Demonstrated effective image classification on the MNIST dataset.
• Successfully discovered models with dynamics in visual disentanglement and Tower of Hanoi tasks.
• Generative models were constructed autodidactically to recover factorial structures and dynamics.

Conclusions:

• The proposed Bayesian framework effectively performs structure learning for discrete generative models.
• Prioritizing data ingestion order via expected free energy enhances model discovery.
• The method shows promise for complex tasks involving latent state recovery and dynamics.