AutoPM3: enhancing variant interpretation via LLM-driven PM3 evidence extraction from scientific literature

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Rare diseases impact over 300 million individuals globally, frequently stemming from genetic variations.
• Interpreting genetic variants, particularly gathering literature-based evidence (e.g., ACMG/AMP PM3), is a significant bottleneck in rare disease diagnosis.
• Current methods for evidence extraction are often complex and time-consuming.

Purpose of the Study:

• To develop an automated method for extracting ACMG/AMP PM3 evidence from scientific literature.
• To leverage open-source large language models (LLMs) for efficient and cost-effective variant interpretation.
• To accelerate the diagnostic process for rare diseases.

Main Methods:

• Introduction of AutoPM3, a novel method employing open-source LLMs for automated PM3 evidence extraction.
• Integration of a Text2SQL-based variant extractor and a retrieval-augmented generation (RAG) module.
• Enhancement of the RAG module with a variant-specific retriever and a fine-tuned LLM for processing tables and text separately.
• Creation of PM3-Bench, a curated dataset of 1027 variant-publication evidence pairs from ClinGen.

Main Results:

• AutoPM3 achieved 86.1% accuracy for variant hits and 72.5% recall for in trans variants on openly accessible data.
• The method outperformed existing approaches, including those utilizing larger models.
• A sequential ablation study confirmed the effectiveness of AutoPM3's core components, particularly the variant-specific retriever and Text2SQL.
• Evidence location was achieved in as little as 76 seconds, demonstrating significant efficiency gains.

Conclusions:

• Open-source LLMs provide an efficient and cost-effective solution for automating genetic variant evidence extraction.
• AutoPM3 significantly reduces the time required for variant interpretation, thereby accelerating rare disease diagnosis.
• The developed method has the potential to improve diagnostic yields and streamline clinical workflows for rare genetic disorders.