Validation of a Novel Statistical Method to Identify Aberrant Patient Logging: A Multi-Institutional Study

Area of Science:

• Medical Education
• Health Professions Education
• Data Analytics in Healthcare

Background:

• Student patient encounter logging is crucial for assessing supervised clinical practice experiences (SCPEs).
• Current manual review of logs by faculty is resource-intensive and its accuracy in reflecting actual patient encounters is unknown.
• A need exists for efficient and accurate methods to identify aberrant logging behavior.

Purpose of the Study:

• To identify and validate a statistical method for detecting aberrant student logging in SCPEs.
• To compare the efficacy of this statistical method against traditional faculty review processes.

Main Methods:

• A multi-institutional study involving 6 physician assistant (PA) programs.
• Utilized Mahalanobis Distance (MD) to statistically identify probable multivariate outliers in student logs.
• Validated the MD method against a gold standard of manual faculty consensus review.

Main Results:

• The MD-based categorization demonstrated high sensitivity (0.846) and specificity (0.766) compared to faculty consensus.
• No significant differences were found in key performance metrics between MD-based and individual program faculty categorization.
• The MD method proved to be a reliable tool for identifying aberrant logging behavior.

Conclusions:

• The Mahalanobis Distance (MD) method provides a statistically sound and less labor-intensive approach to identifying aberrant student logging.
• This automated screening method can help optimize faculty resources and facilitate timely interventions for students.
• Implementing MD-based screening can improve the quality and accuracy of clinical exposure logging during SCPEs.