Unsupervised Extraction of Body-Text from Clinical PDF Documents

Area of Science:

• Medical Informatics
• Natural Language Processing (NLP)
• Data Science

Background:

• Clinical PDF documents contain vital information but extracting body text for NLP tasks is challenging.
• Existing methods for text extraction often require manual intervention or supervised learning.

Purpose of the Study:

• To develop an unsupervised algorithm for automatic body-text extraction from clinical PDF documents.
• To enhance the accessibility of clinical data for downstream Natural Language Processing (NLP) applications.

Main Methods:

• Utilized DBSCAN clustering applied to aggregated page data.
• Leveraged text content and coordinate information for text block organization.
• Employed an unsupervised learning approach to process large volumes of data.

Main Results:

• Achieved high precision scores ranging from 0.82 to 0.98.
• Demonstrated strong recall scores between 0.62 and 0.94.
• Reported F1-scores from 0.71 to 0.96 across diverse medical specialties.

Conclusions:

• The unsupervised algorithm effectively extracts body text from clinical PDFs.
• The method shows significant potential for improving NLP tasks in the medical domain.
• Future work will focus on dynamic parameter tuning and scaling with larger datasets.