Quality of Tinnitus Information From Generative AI Systems and Web Search: An Expert‑Rated Comparative Study

Area of Science:

• Digital Health
• Artificial Intelligence in Healthcare
• Medical Information Quality

Background:

• Patients increasingly seek health information online, including tinnitus-related queries.
• Generative artificial intelligence (GenAI) systems present a new source of health information.
• The quality and accessibility of information from GenAI and traditional web search require comparative evaluation.

Purpose of the Study:

• To compare the quality of tinnitus information generated by multiple GenAI systems and Google Search.
• To assess accuracy, clarity, relevance, completeness, and usefulness of AI-generated responses.
• To analyze the readability and accessibility of tinnitus information across digital platforms.

Main Methods:

• A cross-sectional comparative study evaluated 6 GenAI systems and Google Search.
• Thirty common tinnitus-related questions were submitted to each platform.
• Six experts rated responses using the QAMAI framework, assessing 5 quality domains.

Main Results:

• Significant differences in overall quality were observed across systems (P<0.001).
• OpenEvidence yielded the highest mean quality score (4.45±0.72), outperforming other GenAI systems and Google Search (2.27±1.12).
• Completeness was the lowest-performing domain across all platforms; higher expert-rated quality correlated with lower readability (postgraduate level vs. 6th-7th grade).

Conclusions:

• GenAI systems generally provide superior tinnitus information quality compared to web search, though completeness issues persist.
• An inverse relationship exists between expert-rated quality and response readability, potentially limiting patient comprehension.
• Continuous evaluation and clinician oversight are crucial for ensuring safe, comprehensive, and accessible patient-facing health information.