Herbify: an ensemble deep learning framework integrating convolutional neural networks and vision transformers for...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Botany

Background:

• Herbal medicine offers a cost-effective, holistic alternative to synthetic drugs, but accurate identification remains a challenge.
• Synthetic drugs often have adverse side effects and growing affordability concerns.
• Renewed interest in traditional herbal remedies necessitates reliable identification methods.

Purpose of the Study:

• To develop a computer vision framework for accurate and automated herb identification.
• To create a novel, high-quality dataset for herb recognition tasks.
• To evaluate the performance of deep learning models for herb classification.

Main Methods:

• Compiled and refined a novel dataset (Herbify) of 6104 herb images across 91 species.
• Standardized data using the Preprocessing Algorithm for Herb Detection (PAHD).
• Employed transfer learning with Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) in an ensemble framework.

Main Results:

• EfficientNet v2-Large achieved an F₁-score of 99.13%.
• An ensemble model (EfficientL-ViTL) combining EfficientNet v2-Large and ViT-Large/16 reached an F₁-score of 99.56%.
• Developed an AI application, 'Herbify', for practical herb identification.

Conclusions:

• The proposed AI system provides a highly accurate and operationally viable solution for herb identification.
• The study demonstrates the significant potential of AI in botanical applications.
• The developed framework addresses key challenges in automated herb recognition.