Circuit explained: How does a transformer perform compositional generalization

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Machine Learning

Background:

• Compositional generalization is key to human cognition but poorly understood in neural networks.
• Prior work demonstrated transformers can achieve simple compositional generalization on sequence tasks.

Purpose of the Study:

• To mechanistically interpret the circuit responsible for compositional generalization in transformer models.
• To understand how neural networks achieve symbolic compositionality.

Main Methods:

• Utilized causal ablations to isolate the specific neural circuit.
• Performed precise activation edits to analyze circuit behavior.
• Investigated disentangled representations of token position and identity.

Main Results:

• Identified a circuit that performs function composition.
• The circuit uses a general token remapping rule based on position and identity, not specific function semantics.
• Activation edits predictably steered model outputs.

Conclusions:

• The identified circuit provides a mechanistic explanation for compositional generalization in transformers.
• This finding offers testable hypotheses for similar mechanisms in large-scale AI models.
• Understanding these circuits can advance both AI and cognitive science research.