Multi-Year Analysis of Demographic Reporting in Ophthalmology Artificial Intelligence Studies

Sex, gender, race, ancestry, and ethnicity data are critical to ensuring an equitable, generalizable artificial intelligence (AI) framework for clinical applications in ophthalmology. The primary question is whether reporting demographic data in ophthalmology AI studies can be improved. This retrospective study investigated AI studies between June 2020 and June 2023. Reporting of sex, gender, race, and ethnicity data was stratified by publication year and institution. A sub-analysis of National Institutes of Health (NIH)-funded manuscripts assessed adherence to the 2020-2021 updated guidance for data reporting in science and medical journals. Adherence was assessed based on terminology used in the published manuscript; it does not indicate that the underlying construct or assessment method was verified. In total, 1005 articles were screened across 15 journals. Of the 268 included studies, 122 studies reported sex demographics, 35 studies reported gender demographics, and 96 studies reported race, ethnicity, or ancestry demographics. In 2020, 46% of studies reported either race, ethnicity, or ancestry. This frequency decreased from 46% in 2020 to 24% in 2021, followed by a modest increase in 2022 to 44%, and a slight decrease in 2023 to 35%. Sex and gender were reported for 53.57% and 10.71% of studies in 2020, 40% and 6% in 2021, 50% and 17% in 2022, and 44% and 17% in 2023, respectively. These results highlight gaps in demographic reporting. To ensure fair clinical tools, AI ophthalmic research can improve data transparency. Academic journals may require the reporting of demographic data per NIH guidelines to better understand AI tool generalizability.