SelfCheck-Eval: A multi-module framework for zero-resource hallucination detection in large language models.

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Machine Learning

Background:

• Large language models (LLMs) demonstrate broad capabilities but struggle with factual accuracy, a significant issue for high-stakes applications.
• Current benchmarks for detecting LLM-generated inaccuracies are limited to general knowledge, neglecting specialized domains requiring high precision.

Purpose of the Study:

• To address the gap in evaluating LLM hallucinations in specialized domains, particularly mathematical reasoning.
• To introduce a new benchmark dataset and a detection framework for assessing mathematical reasoning hallucinations in LLMs.

Main Methods:

• Developed the American Invitational Mathematics Examination (AIME) Math Hallucination dataset for evaluating mathematical reasoning errors.
• Proposed SelfCheck-Eval, an LLM-agnostic, black-box detection framework integrating semantic, specialized, and contextual consistency modules.
• Evaluated existing detection methods and the proposed framework on biographical and mathematical content.

Main Results:

• Existing LLM hallucination detection methods perform well on biographical data but show significant weaknesses in mathematical reasoning tasks.
• The performance gap persists even with advanced techniques like natural language inference (NLI) fine-tuning, preference learning, and process supervision.
• SelfCheck-Eval provides a framework compatible with both open- and closed-source LLMs for specialized detection.

Conclusions:

• Current LLM hallucination detection approaches have fundamental limitations, especially in specialized reasoning tasks.
• There is a critical need for specialized, black-box-compatible detection methods to ensure trustworthy LLM deployment in sensitive areas.
• The AIME Math Hallucination dataset and SelfCheck-Eval framework offer a path towards more reliable LLM evaluation in specialized domains.