Pilot evaluation of a large language model-mediated pre-consultation interview for vestibular disorders

Area of Science:

• Neuroscience
• Artificial Intelligence
• Medical Informatics

Background:

• Efficient history taking is crucial for vestibular diagnosis but limited by time and patient understanding.
• Large language models (LLMs) show potential for structuring interviews and aiding diagnosis in neurotology.
• Evidence for LLM application in neurotology is currently limited.

Purpose of the Study:

• To develop and evaluate a large language model (LLM)-based system for automated pre-consultation interviews in dizziness diagnosis.
• To assess the system's ability to structure interviews and propose differential diagnoses.
• To investigate the feasibility of LLM-mediated patient interviews in vestibular medicine.

Main Methods:

• A prototype interview application with text input and speech output was developed.
• Patients' free speech was transcribed, processed by an LLM (GPT-4/GPT-4o) via API, and audio replies were provided.
• The interview logic aligned with diagnostic criteria, and 19 patients completed the LLM interview before otolaryngologist assessment.

Main Results:

• The LLM system achieved an 88.7% mean completion rate for history items.
• Diagnostic concordance was 47.4% for Top-1 and 78.9% for Top-3 suggestions.
• Higher concordance was observed for Menière's disease and PPPD; communication quality varied across aspects.

Conclusions:

• LLM-mediated patient interviews are feasible in vestibular medicine, offering preliminary evidence of clinical utility.
• The system demonstrated promising diagnostic alignment for common vestibular conditions like Menière's disease and PPPD.
• Larger cohort studies are necessary to establish the clinical applicability of LLMs in neurotology.