Partial Scene Text Retrieval.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Partial scene text retrieval is challenging due to the lack of patch annotations in training data.
• Existing methods are limited to text-line instances, failing to address partial patch retrieval.
• This limitation hinders applications requiring fine-grained text localization and recognition.

Purpose of the Study:

• To develop a network capable of simultaneously retrieving both text-line instances and partial patches from scene images.
• To address the limitations of existing methods in handling partial text instances without requiring explicit patch annotations.
• To enhance the efficiency and accuracy of partial scene text retrieval.

Main Methods:

• A novel network embedding query text and scene text instances into a shared feature space for cross-modal similarity measurement.
• Multiple Instance Learning (MIL) adapted for partial patch similarity learning without extra annotations.
• Ranking MIL (RankMIL) to filter noisy samples and a Dynamic Partial Match Algorithm (DPMA) for efficient inference.

Main Results:

• The proposed method achieves state-of-the-art performance on both English and Chinese datasets for text-line and partial patch retrieval tasks.
• Significant improvements in mean Average Precision (mAP) were observed, with up to 8.04% and 12.71% for English, and 24.45% and 38.06% for Chinese.
• The RankMIL and DPMA components effectively reduce noise and enhance search efficiency.

Conclusions:

• The developed network effectively addresses the challenge of partial scene text retrieval by integrating text-line and partial patch search.
• The novel RankMIL and DPMA approaches offer significant advancements in both accuracy and efficiency for partial text retrieval.
• The method demonstrates strong generalization capabilities across different languages and datasets, paving the way for more robust scene text understanding.