Assessing the User Experience of an LLM-Based Conversational Assistant in Diabetes Mellitus Care

This article presents the design, implementation, and evaluation of MarIA, a GPT-3.5-powered virtual assistant integrated into a messaging platform to support patients with type 2 diabetes mellitus (DM). MarIA employs a multi-agent architecture that enables varying dialogue styles and degrees of personalization. In a 3-month longitudinal study involving 35 participants, personalized interactions increased engagement by 26%, while message length more than quadrupled-yielding a richer understanding of patient context. This deeper contextualization enabled MarIA to initiate more relevant, meaningful conversations, fostering a positive cycle of sustained engagement. Safety was critically assessed. While MarIA did not generate factual hallucinations, some general health suggestions-though accurate in isolation-could be inappropriate for users with specific clinical constraints. This underscores the need not only for comprehensive patient profiling but also for an embedded safety layer capable of detecting potentially unsuitable recommendations before or even after delivery. The multi-agent architecture proved essential in enabling proactive behaviors, nuanced context detection, and dialogue adaptability, ultimately enhancing both engagement and user safety in AI-supported chronic care.

The online version contains supplementary material available at 10.1007/s41666-025-00217-5.