Complexity and Target Preservation in Category Maps

Area of Science:

• Cognitive Science
• Information Theory
• Machine Learning

Background:

• Categorization is often viewed as data compression, reducing complex stimuli to simpler classes.
• However, simple compression doesn't guarantee a category map's utility.

Purpose of the Study:

• Develop an information-theoretic framework to evaluate categorization.
• Assess category maps based on complexity and preservation of target-relevant information.

Main Methods:

• Modeled categorization as a many-to-one mapping from stimuli to labels.
• Quantified category distribution using category entropy.
• Applied the framework to synthetic data and learned visual representations (ResNet-50 on CIFAR-10).

Main Results:

• Category maps can retain significant information about unintended variables.
• The utility of a categorization depends on the specific target variable.
• Learned representations may preserve nuisance information.

Conclusions:

• Categorization effectiveness should be judged by information retained about a specific target.
• Evaluation should consider both compression and target-specific information preservation.
• This framework offers a nuanced approach to understanding categorization in AI and cognition.