Natural language processing models for patient-centered summaries of prostatectomy pathology reports

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Pathology Reporting

Background:

• Patients increasingly access radical prostatectomy (RP) pathology reports before clinical discussion, causing anxiety and administrative strain.
• Effective interpretation of complex medical language in patient-facing reports is crucial with growing AI adoption.
• There is a need for optimized AI frameworks to generate accurate, patient-friendly RP pathology summaries.

Purpose of the Study:

• To compare a rules-based natural language processing (NLP) model and a large language model (LLM) for generating patient-facing RP pathology summaries.
• To evaluate the accuracy of AI models in extracting key pathology features, calculating recurrence-free probabilities, and providing clinical recommendations.
• To identify the optimal AI framework for improving patient understanding of complex medical information.

Main Methods:

• A rules-based NLP model and an LLM using zero-shot prompting were developed and tested on RP pathology reports.
• Model performance was assessed for accuracy in extracting specific data points and generating clinical recommendations.
• The models were evaluated on single-institution data and an external test set from a separate institution.

Main Results:

• Error-free summaries were generated in 92% of rules-based NLP reports and 97% of LLM reports.
• The LLM maintained high accuracy on an external test set without retraining.
• The rules-based NLP model achieved high accuracy on the external set after minimal refinement.

Conclusions:

• Both rules-based NLP and LLM approaches can effectively generate accurate patient-facing RP pathology summaries.
• The choice of AI framework can be tailored to institutional resources, costs, and priorities.
• These AI tools can improve patient comprehension and reduce healthcare system burdens.