OphthoChat: A HIPAA-Compliant, Artificial Intelligence-Driven Natural-Language Chatbot for Ophthalmic Electronic...

To introduce and evaluate OphthoChat, a Health Insurance Portability and Accountability Act-compliant, artificial intelligence (AI)‑powered natural language chatbot designed to enable clinicians to query ophthalmic electronic medical records (EMRs) using conversational language, without requiring coding experience.

Retrospective validation study using a clinical data set and clinician-authored queries.

The study included 500 adult patients seen at Columbia University Irving Medical Center between 2020 and 2025: 250 patients who received intravitreal injections for wet age-related macular degeneration and 250 patients who underwent cataract surgery.

We processed >50 000 EMR artifacts, including progress notes, operative reports, imaging summaries, and optical character recognition‑converted scanned documents, into a vectorized database. OphthoChat used a retrieval-augmented generation architecture powered by GPT-4o to answer 1000 natural-language queries stratified by complexity (simple, intermediate, advanced). Each response was evaluated against a reference standard determined by dual clinician adjudication.

Primary metrics included response accuracy, sensitivity, specificity, and time-to-answer per query. Supporting chart citations were also assessed for traceability and rank.

OphthoChat achieved an overall accuracy of 96%, with a sensitivity of 93% and a specificity of 97%. Median time to answer a query was 35 seconds, an 11-fold speed improvement over manual chart review (6.6 minutes). Among correct responses, 97% of cited lines matched the reference standard, with a mean reciprocal rank of 0.92. Performance remained strong across all difficulty levels, including 91% accuracy on advanced narrative inference tasks. The system demonstrated robustness in processing complex chart histories while offering full traceability.

OphthoChat enables fast, accurate, and traceable chart review in ophthalmology through natural-language dialogue. By eliminating the need for technical setup or programming knowledge, it significantly lowers the barrier to AI adoption in clinical research and practice. OphthoChat's ability to synthesize complex EMR data across structured and unstructured formats offers a scalable solution for accelerating outcomes research, clinical decision-making, and cohort identification. Further development will focus on EMR integration and subspecialty expansion.

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.