Design, Development, and Evaluation of Multimodal Conversational Agents for Health Data Registration and Monitoring:...

Area of Science:

• Human-Computer Interaction
• Digital Health
• Conversational AI

Background:

• Patient-generated health data (PGHD) management is crucial for chronic disease self-monitoring.
• Multimodal conversational agents offer a novel approach to engaging patients with eHealth platforms.
• Existing interfaces may not fully leverage the potential of natural language interaction for health data collection.

Purpose of the Study:

• To propose an implementation-oriented design framework for multimodal conversational agents.
• To evaluate the user experience of conversational data-entry workflows in hypertension self-monitoring.
• To compare text-based and voice-based conversational agents against a conventional app interface.

Main Methods:

• Developed a framework operationalizing social intelligence, communication style, anthropomorphic characteristics, and technological mapping.
• Instantiated the framework in two conversational agents integrated into an eHealth platform.
• Conducted a three-arm, single-session feasibility experiment (n=18) comparing app, text, and voice interfaces, using qualitative analysis and the User Experience Questionnaire (UEQ).

Main Results:

• All modalities were acceptable, with positive UEQ scores.
• Conversational agents showed potential for innovation and engagement, with medium to large effect sizes for user experience metrics like Stimulation and Hedonic Quality.
• Voice-based interactions presented minor, session-resolved modality-specific difficulties (e.g., diction).

Conclusions:

• The proposed framework and feasibility study offer preliminary evidence for multimodal conversational interfaces in healthcare.
• The design dimensions align with user feedback, suggesting their relevance for conversational agent development.
• Further adequately powered longitudinal studies are needed to assess clinical and behavioral outcomes.