Effectiveness of Communication Competence in AI Conversational Agents for Health: Systematic Review and Meta-Analysis

Area of Science:

• Human-Computer Interaction
• Health Communication
• Artificial Intelligence

Background:

• Advancements in AI and large language models drive the need for improved conversational agents (CAs) in healthcare.
• Communication competence, crucial in human interactions, is now extended to human-machine communication for CAs.
• This concept emphasizes CAs' ability to exhibit appropriate communicative behaviors for effective human relationship management.

Purpose of the Study:

• To systematically review the operationalization of communication competence in health-related CAs.
• To assess the impact of communication competence on user evaluations, CA usage, psychological well-being, and health outcomes.
• To identify strategies for enhancing communication competence in health CAs.

Main Methods:

• A systematic literature search across 7 major databases (PubMed, Scopus, Web of Science, etc.).
• Inclusion of experimental studies manipulating CA communication competence with human participants.
• Meta-analysis of 25 studies (4525 participants) using three-level random-effects models; risk of bias assessed with the revised Cochrane tool.

Main Results:

• Eleven strategies enhance CA communication competence (e.g., empathy, personalization, humor).
• Communication competence significantly impacts user evaluations (g=0.45) and psychological outcomes (g=0.49).
• No significant effects were found on CA usage or health outcomes; effects remained stable across moderators.

Conclusions:

• Communication competence is vital for health CAs, positively influencing user perceptions and psychological states.
• Limited evidence currently exists on the direct impact of communication competence on tangible health outcomes.
• Future research should focus on evaluating the effects of communication competence on measurable health improvements.