Interpretable Feature Extraction from Clinical Notes for Sepsis Prediction: Comparing Rule-Based, LLM, and Hybrid...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Decision Support

Background:

• Embedding-based models for sepsis prediction lack interpretability, hindering trust and regulatory acceptance.
• Extracting clinically relevant features from unstructured electronic health records (EHRs) is challenging.

Purpose of the Study:

• To compare three frameworks for interpretable feature extraction from clinical notes for sepsis prediction.
• To evaluate the performance of rule-based, LLM-based, and hybrid approaches in identifying sepsis-related clinical findings.

Main Methods:

• Compared rule-based (M1), zero-shot LLM with retrieval-augmented generation (M2), and a hybrid (M3) framework using MIMIC-III clinical notes.
• Evaluated 19 SOFA, qSOFA, and SIRS-based features, with expert validation on 190 predictions.
• Focused on extracting quantitative measurements in JSON format.

Main Results:

• M1 showed high precision but low recall.
• M2 achieved high recall (0.95) and F1 (0.83) but lower precision (0.74).
• M3 demonstrated superior performance with high precision (0.90), recall (0.94), and F1 (0.92).

Conclusions:

• Hybrid frameworks combining symbolic methods with LLMs offer accurate and interpretable feature extraction for sepsis prediction.
• This approach improves precision while maintaining high recall, facilitating trust and regulatory compliance.
• Quantitative JSON outputs enhance the utility of extracted clinical findings.