Digitalizing English-language CT Interpretation for Positive Haemorrhage Evaluation Reporting: the DECIPHER study.

Area of Science:

• Medical imaging analysis
• Artificial intelligence in radiology
• Natural language processing in healthcare

Background:

• Identifying traumatic intracranial bleeds (ICB+) from head CT scans is crucial for patient care and research.
• Manual review of unstructured free-text CT reports is time-consuming and inefficient.
• Existing AI classification methods are not optimized for emergency department needs.

Purpose of the Study:

• To evaluate the diagnostic classification performance of three methods for identifying ICB+ in head CT reports: text classification (TC), commercial NLP (Clinithink), and a large language model (DECIPHER-LLM).
• To determine if DECIPHER-LLM can significantly reduce CT report review workload while maintaining 100% sensitivity.

Main Methods:

• Manually labeled 898 anonymized head CT reports for the presence or absence of ICB+.
• Trained TC and NLP models using random training and validation sets.
• Developed prompts to train the DECIPHER-LLM for classification.

Main Results:

• DECIPHER-LLM achieved 100% sensitivity and 97.4% specificity for ICB+ detection.
• TC achieved 87.9% sensitivity and 98.2% specificity; Clinithink achieved 75.9% sensitivity and 96.2% specificity.
• Using DECIPHER-LLM with a 10% probability threshold reduced manual review by 85.7% while maintaining 100% sensitivity.

Conclusions:

• DECIPHER-LLM demonstrated superior performance compared to other free-text classification methods for identifying ICB+.
• The LLM approach offers a promising solution for efficient and accurate analysis of head CT reports in emergency settings.