Lexical hints of accuracy in LLM reasoning chains

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Machine Learning

Background:

• Large Language Models (LLMs) fine-tuned with reinforcement learning and Chain-of-Thought (CoT) reasoning show improved performance.
• However, LLMs exhibit poor calibration, often expressing high confidence in incorrect answers on challenging tasks like Humanity's Last Exam (HLE).

Purpose of the Study:

• To investigate if measurable properties of CoT reasoning can serve as reliable, model-internal confidence signals.
• To identify which CoT features best predict model accuracy and calibration.

Main Methods:

• Analysis of three feature classes in CoT: length, sentiment volatility, and lexicographic markers (including hedging terms).
• Evaluation on diverse benchmarks: Humanity's Last Exam (HLE), Omni-MATH, and GPQA-diamond.
• Utilized models: DeepSeek-R1, Claude 3.7 Sonnet, and Qwen-235B-Think.

Main Results:

• Lexical uncertainty cues (e.g., "guess", "stuck", "hard") are highly informative confidence indicators across benchmarks.
• Sentiment shifts offer a weaker, complementary signal.
• CoT length predicts correctness only on intermediate-difficulty benchmarks (Omni-MATH, GPQA), not on harder tasks (HLE).
• Uncertainty signals are more salient than high-confidence markers for detecting errors.

Conclusions:

• Measurable properties of CoT reasoning, particularly lexical uncertainty, provide reliable internal confidence signals for LLMs.
• These findings support a lightweight post-hoc calibration method to enhance LLM reliability by complementing self-reported probabilities.