Cross-Depicted Historical Motif Categorization and Retrieval with Deep Learning

Area of Science:

• Digital Humanities
• Computer Vision
• Art History

Background:

• Historical manuscript dating relies on watermarks, which present significant cross-depiction challenges.
• Varied representations of the same watermark motif hinder traditional pattern recognition.

Purpose of the Study:

• To develop a content-based image retrieval system for categorizing and identifying cross-depicted historical watermarks.
• To leverage deep learning for improved watermark recognition in manuscript studies.

Main Methods:

• Application of deep neural networks for watermark categorization with varying detail levels.
• Evaluation of classification performance using macro-averaged F1-score (88.3%) and Jaccard Index (79.5%).
• Assessment of similarity matching performance on expert-crafted test sets.

Main Results:

• Achieved 88.3% macro-averaged F1-score on a 12-category classification task.
• Demonstrated 79.5% Jaccard Index for multi-label classification across 622 categories.
• Attained 100% Mean Average Precision in finding all relevant super-class results, overcoming cross-depiction issues.

Conclusions:

• Deep learning effectively categorizes and retrieves cross-depicted historical watermarks.
• The developed system significantly assists humanities scholars in manuscript cataloging.
• This approach achieves unprecedented accuracy in watermark similarity matching.