Diagnostic report generation for macular diseases by natural language processing algorithms

To investigate rule-based and deep learning (DL)-based methods for the automatically generating natural language diagnostic reports for macular diseases.

This diagnostic study collected the ophthalmic images of 2261 eyes from 1303 patients. Colour fundus photographs and optical coherence tomography images were obtained. Eyes without retinal diseases as well as eyes diagnosed with four macular diseases were included. For each eye, a diagnostic report was written with a format consisting of lesion descriptions, diagnoses and recommendations. Subsequently, a rule-based natural language processing (NLP) and a DL-based NLP system were developed to automatically generate a diagnostic report. To assess the effectiveness of these models, two junior ophthalmologists wrote diagnostic reports for the collected images independently. A questionnaire was designed and judged by two retina specialists to grade each report's readability, correctness of diagnosis, lesion description and recommendations.

The rule-based NLP reports achieved higher grades over junior ophthalmologists in correctness of diagnosis (9.13±1.52 vs 9.03±1.42 points) and recommendations (8.55±2.74 vs 8.50±2.53 points). Furthermore, the DL-based NLP reports got slightly lower grades to those of junior ophthalmologists in lesion description (8.82±1.84 vs 9.12±1.20 points, p<0.05), correctness of diagnosis (8.72±2.36 vs 9.08±1.55 points, p<0.05) and recommendations (8.81±2.52 vs 9.15±1.65 points, p<0.05). For readability, the DL-based reports performed better than junior ophthalmologists, with scores of 9.98±0.17 vs 9.94±0.25 points (p=0.094).

The multimodal AI system, coupled with the NLP algorithm, has demonstrated competence in generating reports for four macular diseases compared with junior ophthalmologists.