Design, Development, and Evaluation of Multimodal Conversational Agents for Health Data Registration and Monitoring:...

Objectives: This study proposes an implementation-oriented design framework for multimodal conversational agents handling patient-generated health data and reports an exploratory experiment evaluating its instantiation in hypertension self-monitoring, focusing on user experience of conversational data-entry workflows. Methods: The framework operationalizes four complementary dimensions (social intelligence, communication style, anthropomorphic characteristics, and technological mapping) and was instantiated in two agents integrated into an eHealth platform. Each agent supports users by providing prompts, interpreting responses, checking data plausibility, and confirming submission. A three-arm, single-session feasibility experiment (n=18, n=6 per group) compared a conventional app interface with text-based and voice-based conversational agents. Evaluation triangulated three sources of evidence: open-ended qualitative responses analyzed through descriptive content analysis, session-level researcher observation notes, and the User Experience Questionnaire (UEQ) reported descriptively with one-way ANOVA and η2 effect sizes. Results: All three modalities were acceptable to participants and produced UEQ scores in the positive range. Hesitation was observed in 2 of 6 Control participants, 1 of 6 Text participants, and 3 of 6 Voice participants, with self-reports indicating that voice-related difficulties were modality-specific (diction, command phrasing) and resolved within the session. Qualitative themes of acceptability and innovation, perceived effort, and modality-specific facilitators emerged across the corpus. Between-group ANOVAs did not reach statistical significance (p>0.05), as expected for an underpowered design, yet η2 values were medium for Attractiveness, Efficiency, Dependability, and Pragmatic Quality and large for Stimulation and Hedonic Quality, converging with the qualitative innovation and engagement signal in the conversational conditions. Conclusions: The framework and feasibility experiment provide preliminary, hypothesis-generating evidence on the potential of multimodal conversational interfaces in healthcare. However, no clinical, behavioral, or longitudinal outcomes were assessed. The four design dimensions can be tentatively associated with themes recognizable in user discourse, and the observed effect-size pattern motivates adequately powered longitudinal studies that incorporate behavioral and clinical endpoints alongside user experience measures.