ViTSTR-Transducer: Cross-Attention-Free Vision Transformer Transducer for Scene Text Recognition.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Attention-based encoder-decoder architectures are effective for scene text recognition (STR) due to their language modeling capabilities.
• The cross-attention mechanism in these models is computationally intensive, posing challenges for low-resource environments.

Purpose of the Study:

• To propose a computationally efficient, cross-attention-free STR framework that retains language modeling capabilities.
• To introduce ViTSTR-Transducer, a novel architecture inspired by Vision Transformer (ViT) and Recurrent Neural Network Transducer (RNN-T).

Main Methods:

• Developed ViTSTR-Transducer, a novel STR framework integrating Vision Transformer (ViT) and Recurrent Neural Network Transducer (RNN-T) principles.
• Eliminated the computationally expensive cross-attention operation from the decoding process.
• Focused on learning an internal language model without relying on cross-attention.

Main Results:

• ViTSTR-Transducer models demonstrated significantly lower computational costs (FLOPs) and latency compared to baseline attention-based STR models.
• Achieved recognition accuracy comparable to baseline attention-based models.
• Outperformed baseline context-free ViTSTR models in recognition accuracy and delivered competitive results against state-of-the-art (SOTA) methods.

Conclusions:

• The proposed cross-attention-free ViTSTR-Transducer framework offers an efficient alternative for scene text recognition.
• ViTSTR-Transducer effectively balances computational efficiency with high recognition accuracy, making it suitable for resource-constrained settings.
• The framework represents a competitive advancement in scene text recognition technology.