Precision Grounding: augmenting large language models with evidence-based databases for trustworthy genetic variant...

Area of Science:

• Genomics
• Artificial Intelligence
• Clinical Decision Support

Background:

• Large language models (LLMs) show promise in summarizing complex genetic variant information.
• Current methods often lack the precision required for clinical decision-making due to insufficient evidence grounding.
• Accurate interpretation of genetic variants is crucial for diagnosing and treating diseases.

Purpose of the Study:

• To introduce Precision Grounding, a novel method for augmenting LLMs with variant-specific evidence.
• To enhance the accuracy and clinical utility of LLM-generated summaries for genetic variants.
• To reduce hallucinations and improve the reliability of LLM outputs in a clinical context.

Main Methods:

• Proposed Precision Grounding, a query tool integrating expert-selected resources for factual information retrieval.
• Developed CATT, an open-source tool that queries ClinGen, ClinVar, and GenCC databases using Variation IDs.
• Compared Precision Grounding against web-search grounding and retrieval-augmented generation (MedRAG) using 50 expert-selected variants and GPT-4o.

Main Results:

• Precision Grounding significantly outperformed web-search grounding and MedRAG in accuracy (4.76/5) and completeness (4.94/5).
• MedRAG struggled with clinically relevant information, while web-search grounding was less effective than Precision Grounding.
• Precision Grounding substantially reduced clinically significant hallucinations, including incorrect pathogenicity classifications and variant misinterpretations.

Conclusions:

• Precision Grounding offers a superior approach to grounding LLMs for genetic variant summarization.
• The open-source CATT tool facilitates the integration of curated genetic knowledge, minimizing LLM hallucinations.
• This framework enhances variant interpretation accuracy and completeness, showing strong potential for clinical decision-support systems and genomics workflows.