Related Experiment Video
Updated: Jun 21, 2025
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
543
Measuring cognitive effort using tabular transformer-based language models of electronic health record-based audit
Seunghwan Kim1,2, Benjamin C Warner3, Daphne Lew2
1Roy and Diana Vagelos Division of Biology and Biomedical Sciences, Washington University in St. Louis, St. Louis, MO 63110, United States.
Summary
A new metric, action entropy, was developed to measure cognitive effort in electronic health record (EHR) use. This method accurately identifies high-effort tasks like attention switching, potentially aiding in workflow optimization.
Area of Science:
- Medical Informatics
- Cognitive Science
- Human-Computer Interaction
Background:
- Electronic health records (EHRs) are integral to modern healthcare but can impose significant cognitive burden on clinicians.
- Quantifying the cognitive effort associated with EHR-based tasks is crucial for improving usability and clinician well-being.
Purpose of the Study:
- To develop and validate a novel measure, termed action entropy, for assessing cognitive effort during EHR-based work.
- To evaluate the utility of action entropy in distinguishing between high and low cognitive demand scenarios.
Main Methods:
- Utilized EHR audit logs from attending physicians and advanced practice providers (APPs) in surgical intensive care units.
- Trained neural language models (LMs) to predict the next action based on prior sequences.
- Calculated action entropy as the cross-entropy of predicted next actions and validated it against attention-switching events.
Main Results:
- Action entropy was significantly higher during attention-switching scenarios compared to non-switching scenarios (P < .001).
- Specific attention-switching events, such as switching to or from the EHR inbox, showed substantial increases in action entropy for both attendings and APPs.
- The metric demonstrated discriminant validity in identifying situations of high cognitive effort.
Conclusions:
- Action entropy, an LM-based metric, effectively quantifies cognitive burden in EHR workflows.
- The measure shows promise as a screening tool for identifying behavioral action phenotypes associated with increased cognitive load.
- Further validation could lead to its application in optimizing EHR design and clinical workflows.
