PICO-based assessment and categorization of evidence for digital health interventions: an inductive framework...

Area of Science:

• Health Informatics
• Evidence Synthesis
• Digital Health

Background:

• Systematic reviews on digital health interventions (DHIs) often suffer from methodological flaws, hindering clear evidence interpretation.
• The increasing volume of DHI reviews necessitates efficient methods for higher-level syntheses like umbrella reviews and evidence gap maps.
• Rapid, systematic evidence assessment at the abstract level is crucial for managing the growing body of DHI research.

Purpose of the Study:

• To introduce the PICO-based Assessment and Categorization of Evidence for Digital Health Interventions (PACE4DHI) framework.
• To enable efficient, structured screening of systematic reviews and meta-analyses at the abstract level for evidence and gap mapping (EGM).

Main Methods:

• A comprehensive literature search identified systematic reviews and meta-analyses on DHIs published between 2011 and October 2023.
• Thematic analysis of 250 studies informed the development of the PACE4DHI framework, which was validated on 138 articles.
• PRISMA guidelines were followed, and 9,030 titles/abstracts were screened from 21,161 initial results.

Main Results:

• The PACE4DHI framework includes 41 categories across problems, DHIs, settings, outcomes, and evidence levels.
• Abstract-level PICO-categorization and evidence classification showed varying accuracy but consistent results compared to full-text reviews.
• Abstracts provided more detail on problems and interventions than comparators and outcomes, with higher accuracy in predicting conclusive evidence.

Conclusions:

• The PACE4DHI framework offers a systematic and pragmatic approach to accessing and synthesizing DHI evidence.
• It can inform future systematic review methodologies, optimize EGM, and identify research gaps.
• Integration with AI tools can facilitate ongoing evidence screening and support a searchable evidence database.