Improving Factuality by Contrastive Decoding with Factual and Hallucination Prompts

Area of Science:

• Artificial Intelligence
• Natural Language Processing

Background:

• Large language models (LLMs) exhibit advanced capabilities but are prone to generating inaccurate or irrelevant outputs, a phenomenon known as hallucination.
• Ensuring factual accuracy in LLM-generated content is critical for reliable applications.

Purpose of the Study:

• To introduce a novel decoding method, Decoding with Factual and Hallucination Prompts (DFHP), to mitigate hallucination in LLMs.
• To enhance the factual accuracy of LLMs without the need for retraining.

Main Methods:

• The DFHP method employs contrastive decoding to differentiate output probabilities between factual and hallucination prompts.
• This approach assesses LLM performance on multiple-choice and text generation tasks.

Main Results:

• DFHP significantly improves the factual accuracy of LLMs across various model sizes.
• On the TruthfulQA dataset, DFHP enhanced the LLaMA model's factual accuracy by an average of 6.4% for 7B, 13B, 30B, and 65B versions.
• The method demonstrated effectiveness without requiring additional model training.

Conclusions:

• The DFHP decoding method offers a significant improvement in LLM factual accuracy.
• Its high reliability makes it suitable for critical applications such as medical diagnosis and legal case analysis.