ChatGPT's Role in Addressing the Real-World Concerns of Adolescents With Hearing Loss: Insights From...

This study aimed to evaluate ChatGPT-generated responses to psychosocial concerns of hearing-impaired adolescents, assessed by otolaryngologists with lived experiences, exploring artificial intelligence's (AI's) potential as complementary support when professional or peer assistance is unavailable.

Twenty scenario-based inquiries spanning 5 domains (interpersonal communication, education, emotional well-being, future aspirations, and hearing aid use) were submitted to ChatGPT-4o on August 25, 2024. Responses were evaluated by 6 board-certified otolaryngologists with lived experiences of adolescent hearing loss using 7 criteria: overall quality, empathy, medical consensus alignment, appropriateness, inquiry comprehension, harm, and age-appropriate comprehensibility. Ratings were assigned on a five-point Likert scale; inter-rater reliability was assessed using Fleiss' kappa.

The Fleiss' kappa was 0.442. The mean ratings for all criteria exceeded 4.0, except for overall quality. Logistic regression analysis identified high levels of empathy (odds ratio, 7.189; p < .0001) and inquiry comprehension (odds ratio, 4.163; p = .0185) as significant predictors of high-quality responses. The Kruskal-Wallis test revealed significant differences in ratings for the "medical consensus" criterion across domains (p = .0094). Post hoc analysis indicated that responses in the "emotional well-being" domain (median, 4.0) scored significantly lower than those in the "hearing aid use" domain (median, 5.0) for medical consensus (p = .0058).

Incorporating evaluators with lived experiences provided a reliable framework for assessing AI's potential as a complementary support tool for hearing-impaired adolescents. The findings highlight AI's potential for integrating empathy and improved inquiry comprehension into text-based health-care consultations, particularly in contexts lacking professional or peer support.