How to make an app-based program work and show how it works.

Area of Science:

• Digital health
• Implementation science
• Human-computer interaction

Background:

• Randomized control trials (RCTs) are the standard for digital health evaluation but have limitations.
• RCTs often fail to capture real-world usage contexts and technological adaptability.
• Evaluating digital health interventions necessitates methods that accommodate dynamic user-technology interactions.

Purpose of the Study:

• To explore the adaptability and contextual effectiveness of a digital health intervention for HIV self-testing.
• To investigate how user practices and evolving contexts influence the perceived success of digital health tools.
• To propose improved monitoring and evaluation strategies for digital health interventions.

Main Methods:

• Ethnographic research was employed to study the HIVSmart! app-based program.
• Qualitative analysis focused on how the digital health approach functioned across diverse settings.
• Empirical data was gathered on user practices and environmental factors influencing technology use.

Main Results:

• Ease of use for digital health technology is shaped by evolving user practices and diverse user groups.
• The perceived effectiveness of an app changes over time due to adaptations in the technology and its surrounding support systems.
• Digital health tools require continuous monitoring that accounts for fluctuations in technology and its environment.

Conclusions:

• Traditional RCTs are insufficient for assessing the dynamic nature of digital health interventions.
• Monitoring and evaluation frameworks must capture the real-world, adaptive use of digital health tools.
• Recommendations are provided for enhancing implementation science approaches in digital health assessment.