Related Experiment Video
Updated: Sep 21, 2025
06:40
Author Spotlight: Cost-Effective Transcriptomic Drug Screening - Unlocking New Targets
Published on: February 23, 2024
1.5K
A text-mining tool generated title-abstract screening workload savings: performance evaluation versus single-human
Niamh Carey1, Marie Harte1, Laura Mc Cullagh1
1National Centre for Pharmacoeconomics, Old Stone Building, Trinity Centre for Health Sciences, St James's Hospital, Dublin 8, Ireland; Department of Pharmacology and Therapeutics, Trinity Centre for Health Sciences, St James's Hospital, Dublin 8, Ireland.
Journal of Clinical Epidemiology
|June 2, 2022
Summary
The text-mining tool Abstrackr significantly reduced title and abstract screening workload by 67%, without omitting relevant citations. However, it may increase workload in full-text screening due to overestimation of relevance.
Area of Science:
- Biomedical Informatics
- Systematic Review Methodologies
- Health Sciences Research
Background:
- Systematic reviews require extensive title and abstract screening (Stage 1).
- Text-mining tools like Abstrackr aim to streamline this process.
- Evaluating the efficiency and accuracy of these tools is crucial for research.
Approach:
- A systematic review on diffuse large B cell lymphoma treatments was used as a case study.
- Abstrackr-assisted screening was compared against single-human screening.
- Performance metrics included sensitivity, specificity, workload, and time savings.
Key Points:
- Abstrackr reduced Stage 1 workload by 67% (5.4 days) compared to single-human screening.
- High sensitivity (91%) was achieved, with no relevant citations missed.
- Low specificity (72%) and precision (15.5%) were observed due to a high false positive rate.
Conclusions:
- Abstrackr-assisted screening offers significant workload savings in Stage 1 without compromising the inclusion of relevant citations.
- The tool's overestimation of citation relevance may negatively impact the subsequent full-text screening stage.
- Further optimization of text-mining tools is needed to balance workload reduction and accuracy.
