KnowledgeVIS: Interpreting Language Models by Comparing Fill-in-the-Blank Prompts.

Area of Science:

• Computer Science
• Artificial Intelligence
• Natural Language Processing

Background:

• Large language models (LLMs) are increasingly used for various text-based tasks.
• Understanding the internal workings and learned associations of LLMs is crucial for researchers and engineers.
• Existing interpretation methods may not fully capture the nuances of LLM behavior.

Purpose of the Study:

• To introduce KnowledgeVIS, a novel human-in-the-loop visual analytics system.
• To enable intuitive interpretation of language models through fill-in-the-blank sentence prompts.
• To facilitate the analysis of learned associations and model predictions.

Main Methods:

• Developed KnowledgeVIS, a visual analytics system for LLM interpretation.
• Utilized fill-in-the-blank sentences as prompts to elicit model predictions.
• Implemented a novel semantic clustering technique for analyzing predicted words.
• Designed interactive visualizations to explore prediction likelihood, uniqueness, and relationships.

Main Results:

• KnowledgeVIS effectively reveals learned associations between training data and downstream NLP tasks.
• Users can create and test prompt variations, analyze predictions, and discover insights.
• Visualizations aid in identifying prediction likelihood, uniqueness, and patterns across prompts.
• Demonstrated capabilities through expert feedback and use cases involving biomedical knowledge, stereotypes, and fact discovery.

Conclusions:

• KnowledgeVIS provides an intuitive and effective approach to interpreting large language models.
• The system aids in understanding how LLMs learn and apply knowledge.
• It supports the evaluation of LLMs for bias, knowledge, and factual accuracy.