Trading spaces: computation, representation, and the limits of uninformed learning

Area of Science:

• Cognitive Science
• Machine Learning
• Developmental Psychology

Background:

• Many learning problems, termed "type-2 problems," are statistically intractable due to data requiring systematic recoding.
• These problems are common in biological and AI domains like animal behavior and language acquisition.
• Despite intractability, these problems are routinely solved, posing a significant scientific puzzle.

Purpose of the Study:

• To investigate general mechanisms underlying the solution of type-2 problems.
• To explore how complex problems are solved without relying solely on innate biases.
• To understand the trade-offs between representation and computation in learning.

Main Methods:

• Analysis of existing learning mechanisms, including incremental learning, modular connectionism, and representational redescription.
• Conceptual framework contrasting type-1 (tractable) and type-2 (intractable) problems.
• Examination of how prior representational states are exploited to reduce search.

Main Results:

• Type-2 problems are not rare but standard in complex systems.
• General mechanisms, not just task-specific biases, manage type-2 complexity.
• Exploiting previously achieved representations is key to reducing computational search.

Conclusions:

• Nature employs general strategies to solve statistically intractable learning problems.
• Human cognition, including language and culture, represents large-scale adaptations for managing representational complexity.
• Learning is facilitated by maximally exploiting existing states of representation.