PETR: Rethinking the Capability of Transformer-Based Language Model in Scene Text Recognition

Area of Science:

• Computer Vision
• Natural Language Processing
• Machine Learning

Background:

• Transformer-based language models (TLMs) excel in scene text recognition due to parallel reasoning and global relationship capture.
• TLM performance is currently limited by low-quality visual predictions, leading to increased correction burdens and inaccurate language modeling guidance.

Purpose of the Study:

• To enhance the capability of transformer-based language models in scene text recognition.
• To address limitations in visual prediction accuracy and language modeling guidance inherent in current TLM approaches.

Main Methods:

• Proposed a Progressive scEne Text Recognizer (PETR) incorporating a Destruction Learning Module (DLM) and a Language Rectification Module (LRM).
• DLM trains the vision model on destructed images to improve character-wise accuracy and understand patch relationships.
• LRM progressively optimizes word length for improved language guidance, handling challenging cases like distortion and occlusion.

Main Results:

• PETR achieved 1.0% and 0.8% performance improvements on regular and irregular datasets, respectively, compared to parallel transformer methods.
• The method demonstrated state-of-the-art results on both English and Chinese benchmarks.
• PETR introduced only 1.7M additional parameters, indicating an efficient design.

Conclusions:

• PETR effectively enhances transformer-based language models by reducing correction burden and rectifying language modeling guidance.
• The proposed DLM and LRM modules significantly improve scene text recognition accuracy, especially in challenging conditions.
• PETR represents a significant advancement in scene text recognition, offering superior performance with high efficiency.