Developing a Service Quality Index System for AI Health Care Chatbots: Mixed Methods Study

Area of Science:

• Health Informatics
• Artificial Intelligence in Healthcare
• Service Quality Management

Background:

• Artificial intelligence (AI) health care chatbots are increasingly adopted globally.
• Evaluating the service quality of these chatbots is crucial but lacks standardized methods.
• This study addresses the need for a comprehensive evaluation framework.

Purpose of the Study:

• To develop a service quality index system for AI health care chatbots.
• The system is based on the established SERVQUAL framework.
• The goal is to provide a tool for assessing chatbot performance.

Main Methods:

• An initial indicator pool was created from literature review and expert consultation.
• A 2-round Delphi process with 20 experts refined the indicators.
• The Analytic Hierarchy Process (AHP) determined indicator weights.

Main Results:

• A final index system with 5 primary and 17 secondary indicators was established.
• Key quality domains include assurance, reliability, human-likeness, tangibility, and responsiveness.
• Assurance (0.239) and reliability (0.237) received the highest weights.

Conclusions:

• This research pioneers a validated service quality index for AI health care chatbots.
• The index system, adapted from SERVQUAL, offers a robust evaluation tool.
• Findings provide insights for enhancing AI-driven medical consultation services.