Assessing adverse event reports of hysteroscopic sterilization device removal using natural language processing

Area of Science:

• Medical Informatics
• Natural Language Processing
• Patient Safety

Background:

• Adverse event reporting is crucial for device safety.
• Analyzing unstructured text in adverse event reports presents challenges.
• Sterilization device removal reports contain valuable safety information.

Purpose of the Study:

• To develop and implement a natural language processing (NLP) annotation model for device adverse event reports.
• To identify the most frequent patient and device events associated with sterilization device removal.
• To provide a framework for analyzing unstructured adverse event data.

Main Methods:

• Developed an iterative annotation model to extract six categories of information from device removal reports.
• Trained a natural language processing (NLP) algorithm using the developed annotation model.
• Assessed model performance using precision, recall, and F1 score.
• Analyzed 16,535 device removal reports from January 2005 to June 2018.

Main Results:

• The NLP annotation model achieved high performance with an overall F1 score of 91.5% for labeled items.
• The most frequently reported patient event was abdominal/pelvic/genital pain (79.6%).
• The most frequently reported device event was dislocation/migration (19.2%).
• A significant proportion of removals involved additional procedures like hysterectomy or salpingectomy.
• One-fifth of removals occurred more than 7 years after implantation.

Conclusions:

• A robust NLP annotation model was successfully developed for analyzing device adverse event reports.
• The model effectively extracts key information, complementing existing administrative data analyses.
• Findings highlight significant patient and device events associated with sterilization device removal, informing safety protocols.