Related Experiment Video
Updated: Jun 18, 2026
Visualizing Early Infection Sites of Rice Blast Disease Magnaporthe oryzae on Barley Hordeum vulgare Using a Basic Microscope and a Smartphone
Published on: March 17, 2023
Advancing mold identification in the routine laboratory, performance of smartphone-based imaging and a newly
Lukas Weber1, Sarah Brueningk2, Bettina Schulthess1
1Institute of Medical Microbiology, University of Zurich, Zurich, Switzerland.
Abstract:
Mold identification in clinical diagnostics is traditionally labor-intensive and is dependent on expert interpretation. MoldVision is a deep-learning approach that uses smartphone images of mold cultures to automate identification. We analyzed 161 clinical isolates across four common mold genera: Penicillium spp., Aspergillus spp. (with A. flavus and A. fumigatus), Fusarium spp., and Cladosporium spp. Daily images were captured from the top and bottom of culture plates over 5 days using a standardized smartphone setup, generating over 4,000 images. We trained three variations of Visual Geometry Group 16 (VGG16) convolutional neural networks (CNNs) and benchmarked the best-performing model (VGG16 with dual classification heads) against LightGBM models trained on pre-extracted features and human expert assessments at various time points. The best-performing VGG16 model achieved a mean (SD) AUROC of 92.7% ± 1.8% and sensitivity of 68.7% ± 2.6% across all species. Here, the performance in identifying Cladosporium spp. was best (AUROC of 99.9% ± 0.1%, fivefold cross-validation mean and SD). Regarding the evaluations over time, early-stage classification (days 1-2) was challenging (F1-score 38.8% ± 3.5% across all species) but improved significantly on days 3-5 (F1 92.1% ± 2.8% across all species). Compared to experts, MoldVision consistently showed superior performance, particularly in mature cultures, detecting subtle morphological features earlier and more accurately. Our results demonstrate that CNNs integrated with low-cost smartphone imaging can reliably classify mold species in routine diagnostics, outperforming human experts in many cases. This approach offers a practical and scalable solution for laboratories lacking specialized mycology expertise, especially in resource-limited settings.IMPORTANCETimely, accurate mold identification can save lives, yet today it often requires days of growth and scarce expert time. MoldVision offers a practical alternative: use a standard smartphone to photograph routine agar plates and let an AI model aid with species recognition. Trained on thousands of images across 5 days of growth, the system detects tell-tale colony patterns as they appear and, once growth is established (days 3-5), often matches or exceeds human readers. This could shorten reporting, prioritize which cultures need expert review, and extend reliable mycology support to laboratories without specialized equipment or staffing. Because it relies on affordable tools and a standardized imaging workflow, MoldVision is scalable to resource-limited settings. We also share data and code to encourage external validation and improvement, moving this concept toward faster and equitable fungal diagnostics. These findings highlight a practical path to amplify diagnostic capacity where it is needed most.
Insights
MoldVision uses smartphone images and deep learning to automate mold identification, often outperforming human experts. This AI approach offers a scalable solution for clinical diagnostics, especially in resource-limited settings.
Area of Science:
- Medical Mycology
- Artificial Intelligence in Diagnostics
- Computational Biology
Background:
- Traditional mold identification in clinical diagnostics is time-consuming and requires specialized expertise.
- Automating mold identification can improve diagnostic speed and accuracy, crucial for patient outcomes.
- Existing methods often lack scalability and accessibility, particularly in resource-limited environments.
Purpose of the Study:
- To develop and evaluate MoldVision, a deep learning system using smartphone images for automated mold identification.
- To compare the performance of MoldVision against human expert assessments.
- To assess the scalability and practicality of smartphone-based AI for routine mycology diagnostics.
Main Methods:
- Trained Visual Geometry Group 16 (VGG16) convolutional neural networks (CNNs) on over 4,000 smartphone images of 161 clinical mold isolates across four genera.
- Captured daily images from culture plates over 5 days using a standardized smartphone setup.
- Benchmarked the best-performing VGG16 model against LightGBM models and human expert assessments.
Main Results:
- The best VGG16 model achieved a mean AUROC of 92.7% ± 1.8% and sensitivity of 68.7% ± 2.6% across all species.
- Performance improved significantly from early (days 1-2, F1-score 38.8%) to later stages of culture growth (days 3-5, F1 92.1%).
- MoldVision demonstrated superior performance compared to human experts, especially with mature cultures, and achieved near-perfect identification for *Cladosporium* spp. (AUROC 99.9%).
Conclusions:
- Deep learning models integrated with smartphone imaging provide a reliable and accurate method for mold species classification in clinical diagnostics.
- MoldVision offers a practical, scalable, and cost-effective solution that can augment diagnostic capacity, particularly in resource-limited settings.
- This AI-driven approach has the potential to accelerate mold identification, reduce reliance on expert interpretation, and improve patient care.
