Facilitating information extraction without annotated data using unsupervised and positive-unlabeled learning

Area of Science:

• Natural Language Processing
• Machine Learning
• Information Extraction

Background:

• Information extraction (IE) is crucial for distilling data from unstructured text.
• Training IE models for rare entities (<1% prevalence) is challenging due to the need for large datasets with few positive examples.

Purpose of the Study:

• To develop and evaluate a method for efficient rare entity extraction using biased positive-unlabeled (PU) learning.
• To reduce the manual effort required for collecting training data for information extraction models.

Main Methods:

• Combined unsupervised learning with biased positive-unlabeled (PU) learning.
• Developed a binary classifier trained solely on biased PU data.
• Applied the method to rare entity extraction (<0.42%) from medical malpractice documents.

Main Results:

• The PU learning model achieved an area under the precision-recall curve of 0.283 and an F1-score of 0.410.
• Outperformed fully supervised learning methods (AUC 0.022, F1 0.096) in extracting rare entities.
• Demonstrated significant reduction in manual effort for data annotation.

Conclusions:

• The proposed method effectively addresses the challenge of rare entity extraction in narrative texts.
• Biased PU learning offers a viable alternative to traditional supervised methods when positive examples are scarce.
• This approach has the potential to streamline information extraction tasks in various domains, particularly in specialized fields like medicine.