Predicting 30-Day Postoperative Mortality and American Society of Anesthesiologists Physical Status Using...

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Natural Language Processing in Healthcare

Background:

• Accurate perioperative risk assessment is crucial for surgical planning and patient safety.
• Current models often miss nuances in free-text preoperative notes.
• Large language models (LLMs) offer potential for utilizing unstructured clinical data, but factual accuracy is a concern.

Purpose of the Study:

• To evaluate if integrating LLMs with retrieval-augmented generation (RAG) can enhance predictions of 30-day postoperative mortality.
• To assess the impact of LLM-RAG on American Society of Anesthesiologists (ASA) physical status classification using preoperative notes.

Main Methods:

• Retrospective cohort study of 24,491 medical records.
• Utilized LLaMA 3.1-8B language model with RAG for extracting insights from free-text data.
• Compared LLM-RAG performance against baseline machine learning (ML) models (XGBoost, random forest, logistic regression, SVM).

Main Results:

• LLaMA-RAG achieved superior F1-scores for 30-day mortality prediction (0.4663) compared to ML models (XGBoost: 0.4459).
• LLaMA-RAG demonstrated strong performance in ASA classification (micro F1-score: 0.8409), outperforming ML models.
• The model showed high sensitivity in identifying rare, high-risk cases, including ASA Class 5 patients and postoperative deaths.

Conclusions:

• LLM-RAG significantly improves the prediction of postoperative mortality and ASA classification.
• Grounding LLM outputs in domain knowledge via RAG enhances accuracy and interpretability.
• This approach shows promise for real-world clinical decision support in perioperative care.