Neuro-symbolic AI for auditable cognitive information extraction from medical reports

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Data Extraction
• Medical Informatics

Background:

• Large language models (LLMs) like GPT-4 offer free text interpretation but have limitations in reliability, transparency, and privacy for healthcare applications.
• Rule-based AI provides transparency and reproducibility but struggles with unstructured clinical text.
• A hybrid approach is needed to leverage the strengths of both LLMs and rule-based systems for clinical data extraction.

Purpose of the Study:

• To develop and evaluate a neuro-symbolic AI system for autonomous and reliable clinical data extraction from diagnostic imaging reports.
• To combine the text interpretation capabilities of LLMs with the verifiable logic of rule-based systems.
• To assess the system's performance in extracting specific clinical parameters and answering research questions from prostate cancer PET/CT reports.

Main Methods:

• Developed a neuro-symbolic AI integrating GPT-4 with a rule-based expert system via a semantic integration platform.
• GPT-4 identified candidate facts from reports; the expert system verified these against medical rules for deterministic, traceable labels.
• Evaluated on 206 prostate cancer PET/CT reports, extracting 26 parameters and addressing study inclusion, recurrence identification, and PSA levels, compared against physician references.

Main Results:

• Neuro-symbolic AI achieved perfect scores (F1=1.00) for study inclusion and recurrence detection, with 100% PSA accuracy, outperforming GPT-4 alone.
• The system demonstrated an auditable chain of reasoning and matched physician performance.
• The AI successfully intercepted reports with residual identifiers, preventing unintended sensitive data transfer, highlighting its safety features.

Conclusions:

• Neuro-symbolic AI offers a trustworthy solution for automating clinical data extraction from imaging reports, surpassing standalone LLMs.
• This hybrid approach provides a viable pathway for safe and reliable AI implementation in clinical research and healthcare practice.
• The system's auditable reasoning and privacy-preserving capabilities address key limitations of current AI in medicine.