Modeling Categorical Variables by Mutual Information Decomposition.

Area of Science:

• Statistics
• Data Analysis
• Machine Learning

Background:

• Contingency table analysis often involves complex interactions between numerous variables.
• Identifying indispensable variables and their relationships is crucial for building parsimonious and interpretable models.
• Existing methods may struggle with high-dimensional or sparse contingency tables.

Purpose of the Study:

• To propose and validate a novel approach using mutual information (MI) decomposition for contingency table analysis.
• To identify essential variables and their interactions for constructing simplified statistical models.
• To compare the efficacy of MI decomposition against existing state-of-the-art methods.

Main Methods:

• Mutual Information (MI) decomposition was employed to identify associative variable subsets based on multinomial distributions.
• The identified subsets were used to validate parsimonious log-linear and logistic models.
• The approach was evaluated on two real-world datasets: ischemic stroke risk factors and banking credit attributes.

Main Results:

• MI decomposition successfully identified key variable subsets and their interactions.
• The method facilitated the validation of parsimonious log-linear and logistic models.
• Empirical comparisons demonstrated the effectiveness of MI analysis against other leading methods for variable and model selection.

Conclusions:

• Mutual Information (MI) decomposition offers a powerful and novel method for analyzing contingency tables.
• This approach enables the construction of concise and interpretable statistical models from discrete multivariate data.
• The MI analysis scheme provides a robust tool for variable and model selection in complex datasets.