Related Experiment Video
Updated: Apr 18, 2026
Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
Evaluating Methodological Coherence and Evidence Recognition in Digital Health Systematic Reviews: Sample-based
Uwe Buddrus1, Jan-David Liebe1,2
1Professorship Digital Society, Faculty of Business, Economics and Social Sciences, Hochschule Osnabrück, CO-Gebäude, Raum 102, Albrechtstraße 30 a, Osnabrück, 49076, Germany, 49 541 969 ext 7019.
Background:
Despite a growing number of systematic reviews on digital health interventions, many do not sufficiently support the recognition of conclusive evidence. Methodological shortcomings may impede the identification and communication of robust findings. Abstracts are the basis for study selection in systematic reviews and are increasingly used in automated screening processes and rapid assessments.
Objective:
This meta-research study examines to what extent systematic reviews apply methodological standards-particularly the specification of PICO (population or problem, intervention, comparison, and outcome) elements-and how this relates to the likelihood of conclusive evidence recognition. It is based on a random sample and focuses on the assessment at the abstract level, as abstracts are used independently of the specific review choice to screen and select studies for evidence synthesis, making them critical for evidence recognition.
Methods:
Following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, we conducted a comprehensive database search (2011-2023). From 2528 eligible systematic reviews, a random sample of 250 abstracts was analyzed descriptively. Abstracts were assessed for PICO specification and evidence conclusiveness in the context of further study characteristics.
Results:
In total, 48% (119/250) of reviews showed low or very low PICO specification, and 64% (159/250) reported inconclusive or weak evidence. Higher specification of outcomes and problems was moderately associated with conclusive evidence. Beside the formulation of the research question along the PICO scheme, we identified recurring issues in search and screening strategy design (eg, limited database use, vague search terms, and long search periods), restrictive eligibility criteria (eg, exclusive reliance on randomized controlled trials), inconsistent use of quality appraisal tools, and underusage of alternative synthesis methods to hinder evidence recognition.
Conclusions:
Our findings suggest that methodological coherence across all review stages is a necessary condition to ensure conclusions and evidence-informed decisions in the digitalization of health care are both valid and meaningful. A structured PICO-based framework, which is aligned with current research, builds on well-established categories and provides clear and differentiated definitions that may enhance the focus and evidentiary strength of future reviews.
Related Concept Videos
01:11Purpose of Health Records I
Here's a breakdown of how health records serve these purposes:
01:23Methods of Documentation V: CBE
In CBE, healthcare professionals establish predefined standards of practice that define what constitutes...
01:28Criteria for Causality: Bradford Hill Criteria - II
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:30Methods of Documentation VII: EMR
01:18The Scientific Method in Nursing Process
When using research findings to change practice, one must understand the process used to guide a study. The scientific method is a systematic, step-by-step process that supports the data's validity, reliability, and generalizability. As a result, findings can be...