Behavioral Analysis of Information Salience in Large Language Models

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Computational Linguistics

Background:

• Large Language Models (LLMs) demonstrate proficiency in text summarization, a capability reliant on discerning content importance.
• The precise nature of 'salience' internalized by LLMs remains an open research question.
• Understanding LLM salience is crucial for interpreting their decision-making processes in content selection.

Purpose of the Study:

• To develop an explainable framework for deriving and investigating information salience in LLMs.
• To systematically analyze how LLMs prioritize information during summarization tasks.
• To compare LLM-derived salience with human perceptions.

Main Methods:

• Introduced an explainable framework to probe LLM summarization behavior.
• Utilized length-controlled summarization as a behavioral experiment.
• Employed the tracing of 'Questions Under Discussion' to derive a salience proxy.
• Conducted experiments across 13 diverse LLMs and four distinct datasets.

Main Results:

• LLMs possess a nuanced and hierarchical understanding of information salience.
• Salience patterns were found to be generally consistent across different LLM families and sizes.
• LLM behavior demonstrated high consistency in information prioritization.
• The derived notion of salience was not accessible through model introspection.
• LLM salience showed only a weak correlation with human judgments of information importance.

Conclusions:

• LLMs internalize a structured, hierarchical concept of information salience.
• Despite consistent internal patterns, LLM salience is not directly interpretable or aligned with human intuition.
• Further research is needed to bridge the gap between LLM salience and human understanding.