Explainable Connectionist-Temporal-Classification-Based Scene Text Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Connectionist Temporal Classification (CTC) is widely used in scene text recognition (STR) for its efficiency but lacks explainability due to its reliance on 1D sequences.
• Existing 2D attention-based methods offer better accuracy and localization but are computationally intensive, posing latency challenges.

Purpose of the Study:

• To develop a low-latency STR method that provides model explainability through character localization.
• To enhance 1D CTC decoders by enabling them to process 2D spatial information for improved interpretability.

Main Methods:

• A marginalization-based method is proposed to process 2D feature maps, predicting joint probability distributions over height and class dimensions.
• An 'association map' is introduced for character localization and explanation, serving a similar role to cross-attention maps in attention-based models.
• A Vision Transformer (ViT) combined with a 1D CTC decoder (ViT-CTC) architecture is utilized for STR.

Main Results:

• ViT-CTC models outperform state-of-the-art (SOTA) CTC-based methods in recognition accuracy on benchmarks.
• ViT-CTC models achieve up to a 12x speed boost compared to baseline Transformer-decoder models with minimal accuracy reduction.
• Character locations estimated from the association map show strong alignment with ground-truth bounding boxes and cross-attention maps.

Conclusions:

• The proposed marginalization-based method successfully integrates character localization and explainability into 1D CTC decoders for STR.
• ViT-CTC offers a compelling balance of high recognition accuracy, low latency, and enhanced model interpretability for scene text recognition tasks.