Predicting relations between SOAP note sections: The value of incorporating a clinical information model

Area of Science:

• Natural Language Processing
• Clinical Informatics
• Artificial Intelligence in Medicine

Background:

• Physician progress notes utilize Subjective, Objective, Assessment, and Plan (SOAP) sections for clinical documentation.
• The Assessment section synthesizes patient information, while the Plan section outlines diagnostic and management strategies.
• Understanding the relationship between Assessment and Plan sections can reveal insights into clinical reasoning.

Purpose of the Study:

• To develop and evaluate a novel human-in-the-loop pipeline for classifying Assessment-Plan relationships in SOAP notes.
• To leverage a clinical information model integrating entailment logic and problem-oriented medical records for entity labeling.
• To fine-tune a RoBERTa-large model for the specific task of classifying Assessment-Plan relationships within clinical notes.

Main Methods:

• A human-in-the-loop pipeline was employed for classifying Assessment-Plan relationships in SOAP notes.
• A clinical information model was constructed to label named entities (problems, symptoms, events, complications) across all SOAP sections.
• Iterative training of Named Entity Recognition models and fine-tuning of a RoBERTa-large model were performed.

Main Results:

• The developed model achieved a maximum macro-F1 score of 82.31%, securing top-2 performance in the n2c2 2022 Challenge.
• Post-challenge improvements incorporating Subjective and Objective section annotations led to outperforming the challenge's top model.
• Shapley additive explanations indicated that the language model often relies on shallow heuristics, highlighting areas for future research in knowledge integration.

Conclusions:

• The human-in-the-loop pipeline effectively classifies Assessment-Plan relationships in clinical notes, demonstrating significant potential for analyzing clinical reasoning.
• The study underscores the importance of robust clinical information models and advanced language models for accurate interpretation of medical text.
• Further research is needed to enhance language model clinical logic and knowledge integration for more sophisticated clinical reasoning analysis.