Attention heads of large language models.

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Neuroscience

Background:

• Large language models (LLMs) exhibit human-like performance in complex tasks but lack transparency.
• Understanding LLM internal reasoning, particularly attention mechanisms, is crucial for identifying limitations.
• Attention heads are vital for LLM reasoning and may mirror human cognitive functions.

Purpose of the Study:

• To demystify the internal reasoning processes of LLMs by exploring attention heads.
• To categorize the functions of specific attention heads using a human thought process framework.
• To analyze methodologies for discovering and evaluating attention head functions.

Main Methods:

• Introduced a four-stage framework based on human cognition.
• Reviewed existing research to identify and categorize attention head functions.
• Analyzed experimental methods, evaluation techniques, and benchmarks for attention head research.

Main Results:

• Identified and categorized diverse functions of specific attention heads within LLMs.
• Summarized current experimental approaches for uncovering specialized attention heads.
• Highlighted the role of attention heads in LLM reasoning capabilities.

Conclusions:

• Attention heads are critical components for LLM reasoning, offering insights into their "black-box" nature.
• A structured framework aids in understanding and categorizing attention head roles.
• Further research is needed to fully elucidate attention mechanisms and their limitations.