Effects of AI Assistance Timing on Pharmacists' Trust in Automated Pill Recognition Technology: Within-Participants...

Image-based pill verification systems demonstrate high model accuracy. However, their effectiveness in pharmacy practice depends on how pharmacists interact with the AI output. The timing of AI advice is one design factor that influences these interactions, yet its impact on pharmacists' moment-to-moment trust dynamics during medication verification requires further investigation.

This study aims to investigate how the timing and conditionality of AI assistance shape pharmacists' trust dynamics during medication verification.

Between April and December 2024, 50 licensed pharmacists completed a browser-based simulated medication dispensing task with 2 AI types: ex-ante advice (AI advice given concurrently with clinical information) and ex-post advice (AI advice given after an initial diagnosis). Ex-post advice was further divided into the involved ex-post and the not-involved ex-post conditions. The experiment used a within-participants design with varying AI types and AI recognition patterns (right fill-correct recognition, right fill-incorrect recognition, wrong fill-correct recognition, and wrong fill-incorrect recognition). The primary outcomes were trust adjustment magnitude and trust adjustment, which were analyzed using mixed-effects linear regression models.

Trust adjustment magnitude differed significantly across AI assistance conditions. The involved ex-post condition led to the highest magnitude of trust adjustment, followed by ex-ante advice (mean difference 9.65, 95% CI 8.55-10.75; P<.001), with the not-involved ex-post condition showing the lowest magnitude (mean difference 3.33, 95% CI 2.63-4.03; P<.001). Analysis by each recognition pattern revealed significant differences in trust adjustment when the right drugs were incorrectly rejected. In this pattern, the involved ex-post condition led to larger trust decrements than both ex-ante advice (mean difference -2.13, 95% CI -3.83 to -0.44; P=.008) and not-involved ex-post conditions (mean difference -7.32, 95% CI -13.69 to -.95; P=.009). A marginal difference was observed between ex-ante advice and not-involved ex-post conditions (mean difference -5.19, 95% CI -11.55 to 1.17; P=.051). No significant differences were observed for other recognition patterns.

Both the timing and conditionality of AI assistance influenced pharmacists' trust dynamics. Disagreement-based AI interventions (involved ex-post) that incorrectly challenged pharmacists led to a substantial trust decrement, whereas the not-involved AI intervention resulted in more stable trust fluctuations. These findings highlight the importance of designing AI systems that align intervention strategies with user expertise and task demands to foster appropriate trust in safety-critical workflows.