LEAVS: An LLM-based Labeler for Abdominal CT Supervision.

Area of Science:

• Medical Imaging and Artificial Intelligence
• Radiology Report Analysis
• Natural Language Processing in Healthcare

Background:

• Automated extraction of structured labels from radiology reports aids in developing vision models for abnormality detection.
• Existing research primarily focuses on chest radiology, with limited work on abdominal reports due to anatomical complexity and diverse pathologies.
• There is a need for robust methods to extract detailed information from abdominal CT reports for downstream AI applications.

Purpose of the Study:

• To introduce LEAVS (Large language model Extractor for Abdominal Vision Supervision), a novel system for extracting structured labels from abdominal CT radiology reports.
• To annotate the certainty of presence and urgency of seven types of abnormalities across nine abdominal organs.
• To enable the training of vision models for classifying abdominal organs as normal or abnormal using extracted labels.

Main Methods:

• Utilized a specialized chain-of-thought prompting strategy with a locally-run Large Language Model (LLM).
• Employed sentence extraction and multiple-choice questions within a tree-based decision system for label extraction.
• Focused on abnormalities encompassing most finding types in CT reports for broad coverage.

Main Results:

• Achieved an average F1 score of 0.89 for extracting various abnormality types across abdominal organs, significantly outperforming competing labelers and human performance.
• Demonstrated comparable performance to human annotations for extracting urgency labels.
• Showcased the utility of extracted abnormality labels in training a single vision model for organ classification (normal/abnormal).

Conclusions:

• LEAVS effectively extracts detailed abnormality and urgency information from abdominal CT radiology reports.
• The developed system significantly advances automated analysis of abdominal radiology data.
• The released code and annotations facilitate further research in AI-driven abdominal imaging analysis.