Related Experiment Video

Updated: Jan 16, 2026

A Novel Digital Platform for a Monitored Home-based Cardiac Rehabilitation Program

A Novel Digital Platform for a Monitored Home-based Cardiac Rehabilitation Program

Published on: April 19, 2019

Considerations for evaluating pragmatic design elements in digital health intervention trials: the case of Keep It

Benbow Nanette^1,2,3,4, Li Dennis H^5,6,7,8, Macapagal Kathryn^5,6,7

¹Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA. nanette@northwestern.edu.

Implementation Science Communications

|October 4, 2025

Abstract

Background:

Digital health interventions are increasingly promoted in healthcare and prevention practices due to their potential for reaching key populations in a cost-efficient manner. Yet there has been limited research on how to effectively implement them with pragmatic approaches that can facilitate scale-up. Keep It Up! (KIU!) 3.0 was a hybrid type 3 implementation-effectiveness trial comparing two delivery strategies (i.e. trial arms) of an HIV prevention intervention for cisgender, young men who have sex with men. We aimed to determine the level of pragmatism of our two-armed trial before and after changes to the county-randomized design.

Methods:

We applied different versions of the PRagmatic Explanatory Continuum Indicator Summary (PRECIS) tool to the two trial arms: delivery of KIU! by community-based organizations (CBO) versus centralized, direct-to-consumer (DTC) delivery. We scored PRECIS-2 for the original study design and the modified design in which the DTC strategy expanded nationally. We applied PRECIS-2-PS to the modified study design. Nine coders in three groups independently scored the tools. Scores were iteratively discussed to arrive at one consensus score per domain, tool, design stage, and arm. We plotted results using the PRECIS-2 and PRECIS-2-PS wheels and averaged domains scores to describe overall score along the Pragmatic-Explanatory Continuum.

Results:

Using PRECIS-2, the trial was on the pragmatic side of the spectrum for both arms and design stages, with average ratings ranging from 3.89-4.33. Both arms were highly pragmatic in the original and modified design in the Setting and Primary Analysis domains and least pragmatic in the Follow-up domain. In the modified trial design, the CBO and DTC arms again scored rather pragmatic using the PRECIS-2-PS tool, but CBO arm scored higher in the eligibility, recruitment, and organization domains compared to PRECIS-2 (5 vs. 4, respectively).

Conclusions:

Application of both the PRECIS-2 and PRECIS-2-PS tools validated the pragmatic design of KIU! 3.0 as originally designed and after modifications during trial implementation. Our findings highlight instances where one tool may be more suitable than the other to assess the pragmatic-explanatory continuum for emerging digital health interventions delivered in diverse settings and with different implementation strategies.

Keywords:

Digital health interventions HIV PRECIS-2 PRECIS-2-PS Pragmatic trials

More Related Videos

The Dyspepsia Educational Tool As a Novel Aid in Dyspepsia Management

The Dyspepsia Educational Tool As a Novel Aid in Dyspepsia Management

Published on: June 29, 2019

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Published on: May 15, 2020

Related Concept Videos

Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches

Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches

Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...

Study Designs in Epidemiology

Study Designs in Epidemiology

Epidemiological study designs are fundamental tools for investigating the distribution, determinants, and control of health conditions in populations. They help researchers understand the relationships between exposures and outcomes, and they broadly fall into two categories: "observational" and "experimental" studies.
Observational studies are those where the researcher does not intervene but rather observes natural variations. They include cross-sectional, cohort, and...

Clinical Trials

Clinical Trials

Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...

Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs

Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs

Body:Bioequivalence experimental study designs play a pivotal role in testing the effectiveness of various treatments. Key among these are the repeated measures, cross-over, carry-over, and Latin square designs. In the repeated measures design, each subject receives all treatments, allowing for temporal comparisons. This type of design is useful in reducing variability but requires careful planning to avoid bias.The cross-over design, an economical method, involves sequential administration of...

Biopharmaceutical Factors Influencing Drug Product Design: Overview

Biopharmaceutical Factors Influencing Drug Product Design: Overview

Rational drug product design integrates knowledge of the drug’s physicochemical properties, formulation components, manufacturing techniques, and intended route of administration. Each factor influences the drug’s performance, including how it is released, absorbed, and eliminated in the body.The physicochemical properties of a drug—such as solubility, stability, and particle size—affect its compatibility with excipients and the choice of dosage form. Excipients, though...

Clinical Trials: Overview

Clinical Trials: Overview

Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...

Articles linked to this work by shared authors, journal, and citation graph.

Associations between organizational readiness and implementation fidelity in national implementation of WHO's labour care guide in Sweden: a prospective observational study.

Implementation science communications·2026

Implementation barriers, facilitators, and fidelity of a mobile money-based savings tool for pregnant women in the Analamanga region of Madagascar: an ex-post process evaluation.

Implementation science communications·2026

Disproportionate pace of evidence-based care implementation in pediatric-onset multiple sclerosis: consequences and opportunities.

Implementation science communications·2026

Reframing simulation as an implementation strategy: a call for conceptual integration across two scientific communities.

Implementation science communications·2026

A co-creation approach to selecting and adapting adolescent tobacco and air-pollution interventions and strategies for implementation in diverse countries: insights from FRESHAIR4Life.

Implementation science communications·2026

Barriers and facilitators to the implementation of a complex nursing intervention: a human-in-the-loop LLM-Assisted longitudinal and Cross-Case analysis across twelve cardiac surgery centres.

Implementation science communications·2026

See all related articles