Improving Scene Text Recognition for Indian Languages with Transfer Learning and Font Diversity

Area of Science:

• Computer Vision
• Natural Language Processing
• Artificial Intelligence

Background:

• Reading Indian scene text is challenging due to regional vocabulary, diverse scripts, and font variations.
• Existing Scene Text Recognition (STR) systems often struggle with the complexities of Indian languages.
• Current synthetic data generation methods may not adequately capture the full spectrum of Indian font diversity.

Purpose of the Study:

• To investigate performance differences between Indian and Latin Scene Text Recognition (STR) systems.
• To improve the robustness of STR systems for Indian languages by enhancing synthetic data generation.
• To explore the effectiveness of transfer learning among various Indian languages for scene text recognition.

Main Methods:

• Utilizing both Unicode and non-Unicode fonts in synthetic data generation to cover broader font diversity for Indian languages.
• Conducting transfer learning experiments across six different Indian languages using synthetic images with common backgrounds.
• Evaluating system performance on real-world datasets, including IIIT-ILST, MLT-17, and a newly created dataset with Gujarati and Tamil scene images.
• Applying lexicon-based transcription approaches for enhanced accuracy.

Main Results:

• Transfer learning experiments revealed that Indian scripts benefit more from each other than from English datasets.
• Significant performance improvements were achieved on four Indian languages in real-world settings.
• Enriching synthetic data with non-Unicode fonts and augmentations led to over 33% Word Recognition Rate gain on the IIIT-ILST Hindi dataset.
• Lexicon-based transcription results were presented for all six languages.

Conclusions:

• Incorporating diverse non-Unicode fonts in synthetic data is crucial for improving Indian scene text recognition.
• Cross-lingual transfer learning among Indian languages is a promising direction for enhancing STR systems.
• The proposed methods demonstrate significant advancements in recognizing Indian scene text, paving the way for more robust and accurate systems.