Learning from complaints about surgical care: a large language model-assisted sequential methods analysis.

Area of Science:

• Healthcare quality and safety
• Surgical patient outcomes
• Health informatics

Background:

• Patient complaints offer insights into surgical safety beyond traditional metrics.
• Previous analyses of surgical complaints were limited in scope and specialty-specific.
• A scalable approach using large language models (LLMs) is needed to analyze surgical complaints.

Purpose of the Study:

• To characterize themes, perioperative processes, system factors, and outcomes in surgical complaints.
• To utilize a large language model (LLM)-assisted approach for scalable analysis of surgical complaints.
• To identify key areas for improving surgical patient safety based on complaint data.

Main Methods:

• Retrospective, cross-sectional study of 650 surgical complaint reports from New Zealand.
• Employed a fixed-parameter LLM workflow to extract data on demographics, clinical context, complications, and system factors.
• Utilized LLM-supported thematic synthesis for identifying complaint themes, with human validation (Cohen's κ >0.85).

Main Results:

• Postoperative complications were frequent (84.2%), often leading to permanent disability (33.2%) or death (24.2%).
• Delays in recognizing deterioration (76.1%), escalation (58.3%), and management (49.5%) were common and linked to increased mortality.
• Key themes included postoperative management, communication/informed consent, and professional conduct, with technical errors being less frequent.

Conclusions:

• Surgical care complaints primarily highlight issues in postoperative monitoring, escalation, and communication, rather than intraoperative technical errors.
• System improvements should focus on early recognition of deterioration, reliable handover/escalation, and enhanced consent processes.
• LLM-assisted analysis offers a scalable method for analyzing complaints to drive organizational learning and quality improvement in surgery.