The Geometry of Concepts: Sparse Autoencoder Feature Structure.

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Computational Linguistics

Background:

• Sparse autoencoders can represent the conceptual universe of large language models (LLMs) as high-dimensional vectors.
• Understanding the internal structure of LLMs is crucial for interpretability and further development.

Purpose of the Study:

• To investigate the multi-scale structural organization of the concept universe within LLMs.
• To identify and characterize patterns at atomic, intermediate, and large scales.

Main Methods:

• Analysis of feature vector geometry derived from sparse autoencoders.
• Application of linear discriminant analysis to project out distractor directions.
• Quantification of spatial modularity and feature clustering using multiple metrics.
• Examination of the eigenvalue power spectrum and clustering entropy across model layers.

Main Results:

• Identified "crystals" with parallelogram/trapezoid faces at the atomic scale, improving with distractor projection.
• Discovered "brain-like" modularity with spatial clustering of features (e.g., math, code) at the intermediate scale.
• Characterized a non-isotropic "galaxy"-scale structure with a power-law eigenvalue distribution, varying by layer.

Conclusions:

• LLM concept spaces exhibit rich, multi-scale geometric structure.
• This structure is not random and shows modularity analogous to biological systems.
• Findings provide a framework for understanding and potentially manipulating LLM representations.