Assessing the Assessment-Developing and Deploying a Novel Tool for Evaluating Clinical Notes' Diagnostic Assessment...

Area of Science:

• Medical Quality Improvement
• Diagnostic Accuracy
• Patient Safety

Background:

• Diagnostic errors in ambulatory care are a significant quality and safety concern.
• Existing tools for evaluating diagnostic assessments in medical records are insufficient.
• There is a need for standardized methods to assess diagnostic quality.

Purpose of the Study:

• To develop and validate the Assessing the Assessment (ATA) instrument.
• To measure the quality of diagnostic assessment notes in primary care urgent encounters.
• To identify common elements and areas for improvement in diagnostic documentation.

Main Methods:

• Retrospective chart review of urgent care encounters.
• Application of the ATA instrument to electronic health record (EHR) notes.
• Evaluation of the ATA instrument for inter-rater reliability and internal consistency.

Main Results:

• The ATA instrument demonstrated reasonable performance (kappa=0.63, Cronbach's alpha=0.76).
• Significant variability was observed in diagnostic assessments across different domains.
• "Don't miss diagnoses" and "red flag symptoms" were well-documented (84% and 87% respectively).
• Psychosocial context documentation was notably low (18%).

Conclusions:

• The ATA tool provides a promising framework for evaluating diagnostic assessments.
• The instrument can identify specific areas for improvement in clinical documentation.
• Enhancing the documentation of psychosocial context is a key area for future quality improvement efforts.