Bayesian inference of minimally complex models with interactions of arbitrary order

Area of Science:

• Statistical modeling
• Machine learning
• Information theory

Background:

• Analyzing high-dimensional datasets with complex, high-order patterns is challenging.
• Traditional methods often focus on pairwise correlations, missing intricate variable interactions.

Purpose of the Study:

• To develop a computationally feasible method for identifying the best statistical model describing high-dimensional binary data.
• To explore high-order dependencies beyond pairwise correlations using a novel model family.

Main Methods:

• Introduction of minimally complex models (MCMs): maximum entropy models with grouped high-order interactions.
• Development of efficient Bayesian model selection for MCMs, enabling rapid exploration of model space.
• Gauge transformation invariance for representation-independent statistical modeling.

Main Results:

• Bayesian model selection for MCMs is computationally efficient, requiring no parameter fitting.
• The MCM framework allows for fast inference and sampling.
• MCMs effectively reveal high-order dependencies and generate falsifiable predictions.

Conclusions:

• Minimally complex models offer a powerful and efficient approach to modeling complex dependencies in high-dimensional binary data.
• This method simplifies the discovery of intricate patterns and provides insights into system symmetries and invariances.