Comparative Evaluation of ChatGPT, Google Translate, and UD Talk for Chinese-to-Japanese Translation in Cardiology...

Language barriers between health care providers and patients can compromise communication quality, patient safety, and health care equity. When professional interpreter services are limited, particularly in outpatient settings, artificial intelligence-based translation tools may serve as supplementary communication aids.

This study compared 3 Chinese-to-Japanese translation tools (ChatGPT, Google Translate, and UD Talk) in terms of their performance in real-world cardiology and pulmonology outpatient consultations.

In this single-center prospective observational study, audio-recorded outpatient consultations between December 2024 and November 2025 were analyzed. Verbatim physician-patient dialogues were translated using the 3 systems. Selected dialogue exchanges were evaluated by professional medical interpreters for translation accuracy and by Japanese-speaking lay participants for translation satisfaction using a 6-point Likert scale. Similarity of translation outputs was assessed at the dialogue exchange level.

A total of 20 outpatient consultations comprising 2450 dialogue exchanges were analyzed. ChatGPT had significantly higher translation accuracy and satisfaction scores than Google Translate and UD Talk, with median scores of 5.0 (IQR 4.0-5.0) for both outcomes compared with 2.0 (IQR 1.0-3.0) for Google Translate and UD Talk (P<.001), and maintained a consistently high performance in both specialties. Google Translate and UD Talk exhibited substantially higher similarity (2131/2450, 87%) to each other than to ChatGPT in translated outputs (119/2450, 4.9% to 122/2450, 5%, respectively; P<.001).

In Chinese-to-Japanese translations of outpatient medical consultations, ChatGPT demonstrated higher accuracy and user satisfaction than Google Translate and UD Talk. These findings indicate that artificial intelligence-assisted translation may support multilingual clinical communication when used as a complementary aid alongside professional interpreter services.