Hespi: a pipeline for automatically detecting information from herbarium specimen sheets

Area of Science:

• Biodiversity informatics
• Digital botany
• Computational biology

Background:

• Specimen-associated biodiversity data are vital for biological, environmental, and conservation research.
• Current manual data extraction from specimen images is slow and inefficient, hindering large-scale analysis.

Purpose of the Study:

• To develop an automated system, Hespi (herbarium specimen sheet pipeline), for efficient and accurate data extraction from herbarium specimen labels.
• To leverage advanced computer vision and natural language processing techniques for digitizing biodiversity data.

Main Methods:

• Hespi integrates two object detection models for sheet and label component identification.
• It employs optical character recognition (OCR) and handwritten text recognition (HTR) for text extraction.
• Extracted text is validated against taxonomic databases and refined using a multimodal large language model.

Main Results:

• Hespi accurately detects and extracts text from specimen sheets across diverse international herbaria.
• The system successfully classifies label types (printed, typed, handwritten, mixed).
• The modular design enables user customization and integration of specific models.

Conclusions:

• Hespi significantly enhances the efficiency and accuracy of extracting authoritative data from herbarium specimens.
• The tool facilitates large-scale biodiversity data digitization, supporting biological, environmental, and conservation sciences.
• Hespi's adaptable architecture promotes broader adoption and development in natural history collections.