Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Machine learning for identifying Randomized Controlled Trials: An evaluation and practitioner's guide.

Iain J Marshall1, Anna Noel-Storr2, Joël Kuiper3

  • 1King's College London, London, UK.

Research Synthesis Methods
|January 10, 2018
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Regional anaesthesia and chronic postsurgical pain: a systematic review and network meta-analysis.

British journal of anaesthesia·2026
Same author

Cochrane Evaluation of (Semi-) Automated Review Methods (CESAR): Protocol for an adaptive platform study within reviews.

Journal of clinical epidemiology·2026
Same author

Reviewing ecosystems of evidence: synthesising the evidence on the commercial determinants of health from a complex systems perspective.

Systematic reviews·2026
Same author

Variation in stroke survivors' long-term home care use: a South London population-based study.

European stroke journal·2026
Same author

Effects of active travel interventions in the UK on disadvantaged population groups and their potential to reduce health inequalities: systematic review.

Journal of epidemiology and community health·2026
Same author

Temporal Trends and Drivers of Socioeconomic Inequalities in Stroke Care, Survival, and Long-Term Outcomes.

Journal of the American Heart Association·2026

Machine learning (ML) models significantly improve the identification of Randomized Controlled Trials (RCTs) compared to traditional filters. This study offers practical guidance and open-source tools for integrating ML into systematic and rapid reviews.

Area of Science:

  • Biomedical Informatics
  • Artificial Intelligence in Healthcare
  • Evidence Synthesis Methodology

Background:

  • Machine learning (ML) algorithms show high accuracy in identifying Randomized Controlled Trials (RCTs).
  • Practical integration of ML for RCT identification into existing workflows remains unclear.
  • Existing methods for RCT identification may not be optimal for all review types.

Purpose of the Study:

  • To evaluate and compare the performance of ML models (SVM, CNN, ensemble) against traditional database search filters for RCT classification.
  • To provide practical guidance on implementing ML models in systematic reviews and rapid reviews.
  • To offer open-source software for applying ML-based RCT identification methods.

Main Methods:

  • Trained and optimized Support Vector Machine (SVM) and Convolutional Neural Network (CNN) models on the Cochrane Crowd RCT dataset (titles and abstracts).

Related Experiment Videos

  • Evaluated model performance on the external Clinical Hedges dataset.
  • Estimated Area Under the Receiver Operating Characteristic curve (AUROC) for performance comparison.

    • Main Results:

    • ML approaches demonstrated superior discrimination between RCTs and non-RCTs compared to traditional filters across all sensitivity levels.
    • The best-performing ML model achieved an AUROC of 0.987 (95% CI, 0.984-0.989), setting a new benchmark for this task.
    • Guidance provided for high-sensitivity (systematic reviews) and high-precision (rapid reviews) strategies with recommended probability cutoffs.

    Conclusions:

    • ML models offer a significant advancement in accurately identifying RCTs for evidence synthesis.
    • The study provides practical, evidence-based recommendations and tools for adopting ML in clinical research workflows.
    • Optimized ML models can enhance the efficiency and accuracy of literature screening in systematic and rapid reviews.