Capturing Surgical Data: Comparing a Quality Improvement Registry to Natural Language Processing and Manual Chart...

Area of Science:

• Surgical research methodology
• Health informatics
• Data abstraction techniques

Background:

• Collecting accurate operative details is crucial for surgical research but remains challenging.
• Surgeon-entered data in clinical registries and automated methods like natural language processing (NLP) offer potential solutions.
• Manual chart review (MCR) is a traditional method for data extraction from electronic medical records (EMR).

Purpose of the Study:

• To compare the accuracy and efficiency of NLP and MCR against surgeon-entered registry data for determining gross bile spillage (GBS) rates during cholecystectomy.
• To evaluate the sensitivity and specificity of NLP and MCR in identifying GBS.
• To assess the time efficiency of data abstraction using each method.

Main Methods:

• Data on bile spillage rates were abstracted from a surgeon-entered clinical registry (July 2018-January 2019).
• These rates were compared with data extracted from the EMR using NLP and MCR.
• Sensitivity, specificity, and data abstraction times were calculated for each method.

Main Results:

• The registry reported a 24.4% rate of GBS, while MCR identified 15.6% and NLP identified 12.7%.
• MCR had 45% sensitivity and 94% specificity; NLP had 27.2% sensitivity and 92% specificity compared to the registry.
• Data abstraction times were significantly faster for the registry (3 min) and NLP (5 min) compared to MCR (12 hours).

Conclusions:

• Neither NLP nor MCR may fully capture operative details like GBS if not clearly documented in the EMR.
• Clinical registries effectively capture operative details but depend on accurate surgeon data input.
• Further research is needed to improve automated data abstraction methods for surgical research.