A probabilistic automated tagger to identify human-related publications

Area of Science:

• Biomedical Informatics
• Computational Biology
• Medical Literature Analysis

Background:

• Manual Medical Subject Heading (MeSH) indexing for human studies in MEDLINE is time-consuming and may delay literature updates.
• Non-MEDLINE articles often lack consistent indexing, hindering comprehensive literature searches.
• There is a need for automated methods to identify human-related publications for timely and broad literature reviews.

Purpose of the Study:

• To develop and evaluate an automated system for identifying human-related studies.
• To provide an independent method for tagging publications lacking MeSH indexing.
• To assist in the triage of clinical evidence for systematic reviews.

Main Methods:

• A linear support vector machine was trained using text-based features from one million MEDLINE records (1987-2014).
• Features included title, abstract, author name, and journal fields.
• The model was evaluated on MEDLINE records from 2015-2016.

Main Results:

• The automated system achieved high accuracy, with an area under the receiver operating curve of 0.976 and an F1 score of 95% relative to MeSH indexing.
• Manual review showed 73.5% agreement with the automated predictions in cases of disagreement with MEDLINE.
• Predictive scores for PubMed-indexed articles are publicly available, along with a web interface for non-MEDLINE articles.

Conclusions:

• The developed automated system effectively identifies human-related studies with high accuracy.
• This tool enhances the timeliness and breadth of literature searches, particularly for articles lacking MeSH indexing.
• The publicly available scores and web interface support efficient evidence triage for systematic reviews.