Evaluating the Impact on Clinical Task Efficiency of a Natural Language Processing Algorithm for Searching Medical...

Area of Science:

• Clinical Informatics
• Health Information Technology
• Natural Language Processing

Background:

• Information retrieval (IR) from free text in electronic health records (EHRs) presents significant time and complexity challenges for clinicians.
• Natural language processing (NLP) offers a potential solution to enhance EHR search functionality, aiming to improve clinical workflow efficiency and reduce cognitive burden.

Purpose of the Study:

• To evaluate the comparative efficacy of three search functionalities—no search, string search, and NLP-enhanced search—for clinical users performing information retrieval tasks within EHRs.
• To assess the impact of different search methods on task completion speed and accuracy in a simulated clinical environment.

Main Methods:

• A simulated clinical environment was created using EHR research software with three sets of patient notes.
• A prospective crossover study design involved 19 doctors and 16 medical students performing information retrieval tasks under three search conditions (no search, string search, NLP-enhanced search).
• Task completion speed and accuracy were measured, alongside user perceptions of NLP-enhanced search via a feedback survey.

Main Results:

• NLP-enhanced search demonstrated statistically significant improvements in task completion accuracy compared to string search (5.14% increase, P=.02) and no search (5.13% increase, P=.08).
• While NLP-enhanced search and string search yielded similar task completion speeds, both were faster than no search (11.5% and 16.0% increases, respectively).
• A substantial majority of users (93%) perceived NLP-enhanced search as more efficient than string search, citing reduced cognitive load.

Conclusions:

• This study represents the largest evaluation to date of diverse search functionalities for clinical users within realistic EHR workflows.
• NLP-enhanced search significantly improves both the accuracy and speed of clinical EHR information retrieval tasks compared to unassisted browsing.
• NLP-enhanced search offers superior accuracy and reduces the number of required searches compared to basic string matching for EHR data retrieval.