AI-empowered health coaching for university students: A mixed-method process evaluation

Artificial Intelligence (AI)-empowered health coaching (HC) has the potential to enhance HC effectiveness by providing real-time, evidence-based support. However, integrating AI into live HC sessions presents challenges, particularly in retrieval accuracy, usability, and engagement. This study evaluates the feasibility of an AI-empowered HC intervention by examining human-AI interaction within a pilot randomized controlled trial.

A process evaluation was conducted using a mixed-methods approach, combining focus group discussions and post-trial quantitative assessments. We explored (i) health coaches' receptivity to AI-empowered HC before the trial, (ii) their engagement and acceptance of the AI-powered Question/Answer (Q/A) system during the trial, and (iii) their recommendations for improvements.

Health coaches expressed positive attitudes toward the AI-empowered HC model, particularly appreciating its ability to provide factual, evidence-based responses. However, engagement was hindered by challenges such as irrelevant or insufficiently detailed answers, which sometimes disrupted the HC flow. Specific recommendations for improvement included enhanced retrieval accuracy, expanded answer options, a full session history, session summaries, and a more interactive user interface.

While AI-empowered HC holds promise, current limitations in extractive Q/A accuracy must be addressed to improve usability and integration within HC workflows. Future work should focus on refining AI retrieval mechanisms, incorporating personalization features, and expanding its adaptability to diverse HC settings. These findings will inform the development of more user-centered AI-empowered HC model that effectively support human health coaches while maintaining the integrity of client interactions.