Verified localization and pharmacognosy of herbal medicinal plants in a combined network framework

Pharmacognosy from medicinal plants involves the scientific domain of medicinal compounding based on their medicinal properties. Accurate identification of medicinal plants is crucial, especially by examining their leaves. Choosing the wrong plant species for medicinal preparations can have adverse side effects. This study presents a Human-Centered Artificial Intelligence approach for medicinal plant identification, combining a YOLOv7-based Leaf Localizer with a leaf Class Verifier based on DenseNet through a Confidence Score Analyser algorithm. The Confidence Score Analyser ensures reliability by evaluating predicted categories against predefined thresholds, and the ensemble technique through majority voting enhances robustness. An average performance gain of 25.66% sensitivity is observed when comparing the YOLO object detection model with 77.45% precision to the YOLO integrated with the class verifier model with 97.33% precision. Consistent sensitivities are achieved through the ensemble technique, showcasing robustness across diverse scenarios. The final step incorporates automated textual and audio pharmacognosy information about the identified medicinal plant properties and their utility. Real-time applicability as a smart phone application makes this approach invaluable for medicinal plant collectors and experts.