Leveraging code-free deep learning for pill recognition in clinical settings: A multicenter, real-world study of...

Area of Science:

• Artificial Intelligence in Healthcare
• Medical Informatics
• Computer Vision for Drug Identification

Background:

• Medication errors, often due to look-alike/sound-alike drug mix-ups, are a significant patient safety concern.
• Automated dispensing systems are not always feasible in high-volume clinical settings.
• AI-powered pill recognition tools offer a potential solution for medication identification.

Purpose of the Study:

• To investigate the viability of code-free deep learning (CFDL) for developing pill recognition models.
• To evaluate the performance of CFDL-developed models in various deployment scenarios, including multicenter clinical settings.
• To explore challenges and propose solutions for optimizing real-world applicability of AI pill recognition tools.

Main Methods:

• Developed a pill recognition model using Microsoft Azure Custom Vision with a dataset of 26,880 images of 30 common solid oral dosage forms.
• Implemented a testing strategy including online API, offline TensorFlow Lite models (Windows PC, Android app), and real-world clinical assessments.
• Evaluated model calibration, reliance on color features, and device dependency.

Main Results:

• High performance in the development platform (98.7% precision, 95.1% recall).
• Internal testing via API achieved 93.7% precision and 88.96% recall.
• Offline Android application performance reduced to 86.50% precision and 75.00% recall; external clinical testing yielded 83.10% precision and 71.39% recall.

Conclusions:

• CFDL is a feasible and cost-effective approach for developing AI pill recognition systems.
• Model performance is highest via online API, with offline performance requiring further optimization.
• Tailoring models to specific formularies and implementing user-centered approaches are crucial for real-world success.