Automating data abstraction in a quality improvement platform for surgical and interventional procedures

Area of Science:

• Medical Informatics
• Natural Language Processing
• Clinical Data Management

Background:

• Manual data abstraction in quality improvement (QI) programs is time-consuming and resource-intensive.
• The Surgical Care and Outcomes Assessment Program (SCOAP) requires extensive data collection for performance benchmarking.
• Automating data abstraction can enhance the efficiency and scalability of QI initiatives.

Purpose of the Study:

• To describe a text processing system for automating data abstraction in QI programs.
• To evaluate the performance of statistical and rule-based extractors for clinical data.
• To identify sources of error in automated data extraction for surgical procedures.

Main Methods:

• Developed a preprocessing pipeline to segment clinical notes into sections, sentences, and tokens.
• Implemented statistical and rule-based extractors to identify and abstract specific data elements.
• Utilized extracted information as features for the automated data abstraction models.

Main Results:

• Evaluated 25 extractors: 14 statistical and 11 rule-based.
• Statistical extractors achieved an average F1-score of 0.812 (range: 0.571-0.993).
• Rule-based extractors achieved an average F1-score of 0.785 (range: 0.576-0.931).

Conclusions:

• The developed system shows promise for automating clinical data abstraction in QI.
• Error analysis indicated data imbalance and gold standard creation as primary error sources.
• Future work will involve larger, multi-institutional datasets to further refine the system.