Automated Classification of Adverse Events After Hydrogel Perirectal Spacer Insertion for Prostate Cancer Using Large...

Area of Science:

• Medical device safety and performance analysis.
• Application of artificial intelligence in healthcare data.
• Radiation oncology and adverse event monitoring.

Background:

• Prostate radiation therapy often utilizes hydrogel spacers like SpaceOAR to minimize rectal toxicity.
• Accurate and efficient analysis of device-related adverse events (AEs) is crucial for patient safety.
• Manual review of large databases for AEs is time-consuming and resource-intensive.

Purpose of the Study:

• To evaluate the performance of large language models (LLMs) in analyzing adverse events associated with the SpaceOAR hydrogel spacer.
• To compare the accuracy of LLMs against human abstraction in classifying SpaceOAR-related AEs.
• To identify common AEs associated with SpaceOAR using LLM analysis of a large dataset.

Main Methods:

• Extracted 1,455 SpaceOAR reports from the FDA MAUDE database (Jan 2015-Dec 2024).
• Manually abstracted 97 reports to classify AE problems, radiation timing modifications, and severity (CTCAE score).
• Compared the accuracy of three LLM families against human reviewers, with the top-performing LLM (GPT-4o) analyzing the full dataset.

Main Results:

• GPT-4o achieved high accuracy (4.96/5) in identifying AE outcomes, closely matching human reviewers (4.99/5).
• LLM analysis of 1,455 reports identified malpositioned gel (58.7%) as the most common issue.
• Other significant AEs included infection/inflammation/abscess (10.4%), fistula (7.1%), and rectal ulcer (4.7%). ICU care and death were rare (0.1% and 0.3%).

Conclusions:

• LLMs can automate the analysis of device-related AEs, significantly reducing manual effort.
• Serious AEs associated with SpaceOAR highlight the need for ongoing safety surveillance.
• Careful consideration and dynamic monitoring are essential when implementing hydrogel spacers.