Leveraging Open-Source Large Language Models to Identify Undiagnosed Patients with Rare Genetic Aortopathies.

Area of Science:

• Genetics
• Medical Informatics
• Cardiology

Background:

• Rare genetic aortopathies often go undiagnosed due to varied symptoms, leading to severe cardiac events.
• Early genetic testing is crucial for proactive interventions but relies on physician recognition and referral.
• Automated screening is needed to identify patients who don't fit typical diagnostic patterns.

Purpose of the Study:

• To develop and validate an open-source Large Language Model (LLM)-enabled pipeline for recommending genetic testing in rare genetic aortopathies.
• To leverage retrieval augmented generation (RAG) for processing clinical notes and identifying at-risk patients.

Main Methods:

• Developed an LLM pipeline integrating RAG on genetic aortopathy corpora.
• Validated the pipeline using 22,510 patient progress notes from 500 individuals (250 cases, 250 controls) in the Penn Medicine BioBank.
• Assessed pipeline performance using metrics like accuracy, precision, sensitivity, and F1-score.

Main Results:

• The pipeline successfully categorized 425 out of 499 patients.
• Achieved a patient-level recommendation accuracy of 0.852, precision of 0.889, and sensitivity of 0.803.
• Demonstrated strong performance with an F1-score of 0.844 and F3-score of 0.811.

Conclusions:

• The LLM-enabled workflow with RAG shows significant potential for identifying patients needing genetic testing for rare genetic aortopathies.
• Automating the analysis of free-text clinical notes can improve early disease detection and patient outcomes.
• This approach can support undiagnosed patient identification and facilitate timely interventions.