Artificial intelligence in a commercial clinical decision support tool: Insightful or dominated by heuristics?

To systematically evaluate a commercially available myopia risk artificial intelligence (AI) supported clinical decision support screening tool in pediatric patients and to determine the additional insights provided by the AI component.

Eighteen synthetic images of myopic-appearing and 18 non-myopic-appearing pediatric 'subjects' were generated using three commercially available AI software (ChatGPT, Google Gemini, and Reve), and uploaded to Magnifeye, which performs AI-guided image analysis and assessment of environmental risk factors to evaluate output myopia risk. The first analysis was image-based, where we assessed each image but modified the environmental risk to the lowest possible and the highest possible level. The second analysis was scenario-based, where we assessed two 'subjects', and systematically evaluated all combinations of environmental risk.

There were significant differences in eye aspect ratio between myopic and non-myopic groups (squinting). Under low environmental risk conditions, both myopic and non-myopic groups were regarded as low overall risk of myopia. Conversely, high environmental risk led to a high overall risk of myopia. Systematically adjusting the environmental risk factors demonstrated a gradual uptitration of overall myopia risk. Family history of myopia and proximity behaviors represented the highest environmental risk. A non-myopic appearance moderated the overall myopia risk, decreasing it slightly in some situations.

A simple clinical problem with a small number of outcomes in a low-stakes situation could be crudely reverse-engineered. This highlights the notion that some clinical issues may not necessarily require a complex AI system, which may be no more effective than conventional clinical heuristics.