A Scene-Text Synthesis Engine Achieved Through Learning From Decomposed Real-World Data.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Scene-text image synthesis is crucial for training deep neural networks, offering accurate annotation data.
• Existing methods often rely on unsupervised learning due to a lack of suitable datasets, leading to performance limitations.
• Previous approaches explored rule-based generation and learning-based methods on 2D and 3D surfaces.

Purpose of the Study:

• To facilitate research on learning-based scene text synthesis.
• To introduce a novel dataset (DecompST) and a Learning-Based Text Synthesis (LBTS) engine.
• To improve the generation of realistic synthetic scene-text images for downstream tasks.

Main Methods:

• The DecompST dataset was created from public benchmarks, featuring quadrilateral BBoxes, stroke-level masks, and text-erased images.
• A Learning-Based Text Synthesis (LBTS) engine was proposed, comprising a text location proposal network (TLPNet) and a text appearance adaptation network (TAANet).
• TLPNet identifies text embedding regions, while TAANet adjusts text geometry and color to match background context.

Main Results:

• The proposed LBTS engine effectively generates synthetic scene-text images.
• Experiments demonstrated that LBTS produces superior pretraining data for scene text detectors compared to existing methods.
• The integrated TLPNet and TAANet successfully adapt text instances to background scenes.

Conclusions:

• The DecompST dataset and LBTS engine significantly advance the field of scene-text image synthesis.
• The developed approach enables the creation of high-quality synthetic data for training robust scene text analysis models.
• The availability of the dataset and code encourages further research and development in this area.