MAGNET: Counterfactual samples synthesizing for mitigating hallucination in large language models

Area of Science:

• Artificial Intelligence
• Natural Language Processing

Background:

• Large language models (LLMs) often exhibit hallucinations, a significant drawback impacting their reliability.
• Existing methods to reduce hallucinations have limitations, particularly concerning biases inherited from pre-training data's co-occurrence statistics.

Purpose of the Study:

• To introduce a novel fine-tuning framework, Model-AGNostic countErfacTual synthesis and adaptive fine-tuning (MAGNET), designed to mitigate LLM hallucinations.
• To address and reduce the impact of co-occurrence statistics bias present in pre-training corpora on sentence generation.

Main Methods:

• MAGNET generates counterfactual sample sentences and associated subject/object information from the LLM itself.
• A filtering process ensures generated samples contain specific information before being used for fine-tuning.
• The framework utilizes both original sentences and their generated counterfactual counterparts as a training dataset for adaptive fine-tuning.

Main Results:

• Application of MAGNET to the GPT-Neo 2.7B model resulted in a 12% improvement in the Factual Knowledge Probing experiment.
• Correlation analysis demonstrated MAGNET's ability to mitigate bias originating from pre-training data.
• Fine-tuning the GPT-Neo 125M model on the LAMA-TREx dataset using MAGNET showed a 2.27% performance increase in the TruthfulQA benchmark.

Conclusions:

• MAGNET offers an effective approach to reducing hallucinations in LLMs by tackling biases in pre-training data.
• The framework enhances factual accuracy and truthfulness, demonstrating its practical utility in improving LLM performance.