PICO to PICOS: Weak Supervision to Extend Datasets with New Labels

Area of Science:

• Natural Language Processing
• Clinical Informatics
• Machine Learning

Background:

• Manual annotation of clinical text corpora is expensive and inflexible.
• Extracting PICO (Participant, Intervention, Comparator, Outcome) information aids systematic reviews but often requires additional entity extraction.
• Extending corpora to new entities like study design necessitates manual re-annotation.

Purpose of the Study:

• To adapt Snorkel's weak supervision methodology for extending clinical corpora to new entities without extensive manual labeling.
• To enrich the EBM-PICO corpus with "Study type and design" entities.
• To demonstrate a cost-effective and flexible approach to clinical data annotation.

Main Methods:

• Utilized Snorkel's weak supervision framework to programmatically label data.
• Focused on extracting "Study type and design" as an example entity.
• Applied the methodology to the EBM-PICO corpus, labeling 4,081 documents.

Main Results:

• Achieved programmatic labeling of 4,081 clinical documents using weak supervision.
• Obtained an F1-score of 85.02% for the extraction of "Study type and design" on the test set.
• Demonstrated the feasibility of extending clinical corpora efficiently.

Conclusions:

• Weak supervision offers a scalable solution for expanding clinical corpora.
• The adapted methodology reduces the cost and time associated with manual annotation.
• This approach facilitates more comprehensive data extraction for clinical research and systematic reviews.