Evaluating clinical AI summaries with large language models as judges

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Natural Language Processing

Background:

• Electronic Health Records (EHRs) generate extensive clinical data, posing challenges for healthcare providers to synthesize efficiently.
• Generative AI and Large Language Models (LLMs) offer potential for summarizing EHRs to reduce provider cognitive load.
• Ensuring the accuracy of AI-generated summaries necessitates robust evaluation methods, as human review is time-consuming and expensive.

Purpose of the Study:

• To introduce and validate an automated LLM-based method for assessing the quality of multi-document summaries derived from real-world EHR data.
• To benchmark the LLM-as-a-Judge framework against the established Provider Documentation Summarization Quality Instrument (PDSQI).

Main Methods:

• Developed and validated an LLM-as-a-Judge framework for evaluating EHR multi-document summaries.
• Benchmarked the LLM-based evaluation against human review using the PDSQI.
• Assessed inter-rater reliability and performance metrics including intraclass correlation coefficient (ICC) and evaluation time.

Main Results:

• The LLM-as-a-Judge framework demonstrated strong inter-rater reliability, comparable to human evaluators.
• GPT-o3-mini achieved an ICC of 0.818 and a median score difference of 0 from human assessments, completing evaluations in 22 seconds.
• Reasoning LLMs outperformed other approaches in inter-rater reliability, especially for complex evaluations requiring domain expertise.

Conclusions:

• Automated LLM-based evaluation provides a scalable and efficient method for assessing the accuracy and safety of AI-generated clinical summaries.
• The LLM-as-a-Judge approach can significantly reduce the burden and cost associated with traditional human review.
• This technology facilitates the reliable deployment of AI for synthesizing clinical data in Electronic Health Records.