Crafting Interpretable Embeddings for Language Neuroscience by Asking LLMs Questions

Area of Science:

• Neuroscience
• Artificial Intelligence
• Computational Linguistics

Background:

• Large language models (LLMs) offer advanced text embeddings but lack interpretability, crucial for scientific applications like neuroscience.
• The increasing use of LLMs in science necessitates methods for understanding their internal representations.

Purpose of the Study:

• To investigate if interpretable embeddings can be generated from LLMs using prompting.
• To introduce and evaluate a novel method for creating interpretable embeddings called question-answering embeddings (QA-Emb).

Main Methods:

• Developed QA-Emb, where each feature corresponds to a yes/no answer from an LLM to a specific question.
• Trained QA-Emb by selecting relevant questions rather than optimizing model weights.
• Applied QA-Emb to predict functional magnetic resonance imaging (fMRI) voxel responses to language stimuli.

Main Results:

• QA-Emb significantly outperformed a known interpretable baseline in predicting fMRI responses.
• The method achieved high performance with a minimal set of questions.
• QA-Emb models could be efficiently approximated by a simpler model.

Conclusions:

• QA-Emb provides a flexible and interpretable approach to generating feature spaces for semantic brain representations.
• This method facilitates the concretization and evaluation of our understanding of how the brain processes language.
• QA-Emb shows promise for broader applications in natural language processing (NLP) tasks.