Complexity and Target Preservation in Category Maps

Area of Science:

• Cognitive Science
• Information Theory
• Machine Learning

Background:

• Categorization is often viewed as data compression, reducing complex stimuli to simpler classes.
• However, compression alone doesn't guarantee a category map's utility.
• Existing frameworks lack a comprehensive evaluation of information preservation.

Purpose of the Study:

• Develop an information-theoretic framework to evaluate categorization.
• Assess category maps based on complexity and target-relevant information.
• Analyze how learned visual representations preserve different types of information.

Main Methods:

• Defined categorization as a many-to-one mapping from stimuli to labels.
• Introduced category entropy to quantify label distribution.
• Used synthetic data and a pretrained ResNet-50 on CIFAR-10 for analysis.
• Implemented nuisance controls to test information preservation.

Main Results:

• Category maps can preserve information not aligned with the primary target.
• The utility of a categorization depends on the specific target variable.
• Learned visual representations show varying degrees of information preservation across layers.

Conclusions:

• Categorization effectiveness should be measured by information retained about a specific target.
• Evaluation should consider both compression and target-specific information preservation.
• This framework offers a more nuanced approach to understanding categorization in artificial and biological systems.