ConceptViz: A Visual Analytics Approach for Exploring Concepts in Large Language Models.

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Data Visualization

Background:

• Large language models (LLMs) demonstrate advanced capabilities but their internal knowledge representation remains opaque.
• Sparse Autoencoders (SAEs) offer a method for feature extraction from LLMs, yet their outputs lack direct human interpretability.
• Bridging the gap between SAE features and human-understandable concepts is crucial for advancing LLM interpretability.

Purpose of the Study:

• To introduce ConceptViz, a visual analytics system designed to facilitate the exploration and interpretation of concepts within LLMs.
• To enable researchers to query SAE features using human concepts and validate identified correspondences.

Main Methods:

• ConceptViz employs a novel Identification ⇒ Interpretation ⇒ Validation pipeline.
• Users can query SAEs with specific concepts, interactively explore feature alignments, and verify these alignments through model behavior.

Main Results:

• ConceptViz streamlines the discovery and validation of meaningful concept representations in LLMs.
• The system aids researchers in developing more accurate mental models of LLM features.
• Demonstrated effectiveness through usage scenarios and a user study.

Conclusions:

• ConceptViz significantly enhances the interpretability research process for LLMs.
• The system facilitates a deeper understanding of how LLMs represent knowledge.
• Publicly available code and user guide promote further research and adoption.