Causal networks guiding large language models: application to COVID-19

Area of Science:

• Computational biology
• Artificial intelligence in healthcare
• Medical informatics

Background:

• Causal Networks offer a structured approach to understanding disease diagnosis.
• Large Language Models (LLMs) show potential in processing clinical information.
• Integrating these approaches could enhance diagnostic accuracy.

Purpose of the Study:

• To convert a Causal Network into a Large Language Model (LLM) for COVID-19 diagnosis.
• To compare the diagnostic accuracy of the Causal Network and the LLM.
• To investigate the impact of direct and indirect symptom data on model performance.

Main Methods:

• Developed prompts and completions to translate a Causal Network into an LLM.
• Utilized full-factorial combinations of significant variables for prompt generation.
• Evaluated model accuracy using Area Under the Receiver Operating Curve (AUROC) on two patient databases.

Main Results:

• The Causal Network (AUROC=0.91) was more accurate than the LLM (AUROC=0.88) when indirect symptom information was available.
• Both models showed reduced accuracy (AUROC ~0.75) when indirect information was absent.
• LLM performance was influenced by both reported and unreported data.

Conclusions:

• Causal Networks provide a robust framework for diagnostic modeling.
• LLMs require access to comprehensive data, including indirect predictors, for optimal performance.
• Future conversational LLMs should proactively query for missing indirect information to improve diagnostic capabilities.