The Evidence Aggregator: AI reasoning applied to rare disease diagnostics

Variant assessment in rare disease (RD) diagnostics depends on using domain knowledge in the time-intensive process of retrieving, reviewing, and synthesizing clinical and technical information.

To address these challenges, we developed the Evidence Aggregator (EvAgg), an open-source, generative artificial intelligence-based tool designed to support RD diagnosis that systematically extracts relevant information from the scientific literature for any human gene. Furthermore, we constructed an expert-curated data set and evaluated EvAgg's performance for the tasks of relevant article selection, finding observations of human genetic variation within those articles, and extracting specific details about those observations (eg, zygosity, variant inheritance, variant type, functional study, phenotype, and study type). A user study evaluated the utility and user experience in RD case analysis.

Our evaluation study revealed that EvAgg achieved 92% recall in identifying relevant articles, 96% recall in detecting instances of genetic variation within those articles, and approximately 80% accuracy in extracting individual case and variant-level content. Our subsequent user study evaluated the utility and user experience in RD case analysis. We found that EvAgg reduced review time by 34% (P < .002) and increased the number of articles, variants, and cases evaluated per unit time.

EvAgg provides a thorough and current summary of observed genetic variants and their associated clinical features, supporting the process of manual literature review and enabling rapid synthesis of evidence concerning gene-disease relationships. The demonstrated time savings have the potential to reduce diagnostic latency and increase solve rates for challenging RD cases.