Related Experiment Video
Updated: Jun 5, 2025
12:18
A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
Published on: January 11, 2020
7.5K
Development and Validation of a Literature Screening Tool: Few-Shot Learning Approach in Systematic Reviews.
Phongphat Wiwatthanasetthakarn1, Wanchana Ponthongmak1, Panu Looareesuwan1
1Department of Clinical Epidemiology and Biostatistics, Faculty of Medicine Ramathibodi Hospital, Mahidol University, Bangkok, Thailand.
Journal of Medical Internet Research
|December 11, 2024
Summary
This study introduces a few-shot learning (FSL) framework using S-BERT to streamline systematic review (SR) screening, reducing workload by over 50% but with a potential for missing eligible studies.
Area of Science:
- Machine Learning Applications
- Evidence-Based Medicine
- Biomedical Informatics
Background:
- Systematic reviews (SRs) are crucial for evidence-based medicine but face challenges due to time-intensive literature screening.
- Rapid medical advancements can quickly render SRs outdated, necessitating efficient screening methods.
- Few-shot learning (FSL) and Sentence-BERT (S-BERT) offer promising solutions for streamlining SR study selection with limited data.
Purpose of the Study:
- To develop and validate a model framework utilizing FSL for efficient study screening in SRs.
- The goal was to reduce the workload associated with literature screening while maintaining high recall rates.
- To assess the feasibility and performance of FSL in accelerating the SR process.
Main Methods:
- A novel FSL model framework was developed and validated using S-BERT with titles and abstracts from 9 prior SR projects.
- Key metrics including workload reduction and cosine similarity were used to determine optimal training data size (4-12 studies).
- Prospective evaluation in 4 ongoing SRs compared FSL screening against a secondary reviewer and a gold standard principal reviewer to estimate false negative rates.
Main Results:
- The FSL model achieved significant workload reduction (51.11% to 97.67%) with optimal training sets of 4-6 eligible studies.
- Similarity thresholds for 100% recall ranged from 0.432 to 0.636.
- In prospective evaluations, the FSL model's false negative rate (1.87%-12.20%) was generally lower than a secondary reviewer's (5%-56.48%) compared to the principal reviewer.
Conclusions:
- The developed FSL framework shows significant potential for reducing systematic review screening workload by over 50%.
- The model achieved substantial workload reduction but did not guarantee 100% recall, indicating a risk of omitting relevant studies.
- Future development should focus on creating a web application to enhance accessibility and implementation of the FSL framework for researchers.
