DiffEraser: Generalized Text Erasure Based on Latent Diffusion Prior.

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Scene text removal (STR) models struggle with document images due to complex backgrounds.
• Existing methods fail to accurately estimate background features in text regions, especially in cross-domain scenarios.

Purpose of the Study:

• To introduce DiffEraser, a novel framework for text removal in both scene and document images.
• To leverage Latent Diffusion Model (LDM) prior knowledge for improved background reconstruction.

Main Methods:

• Developed DiffEraser, incorporating a Diffusion-Prior (DP) encoder and a Latent-Fusion (LF) decoder.
• Integrated LDM prior knowledge with image features in the latent space.
• Constructed the NPID295 dataset for evaluating cross-domain performance.

Main Results:

• DiffEraser significantly outperforms existing STR methods on the challenging NPID295 document image dataset.
• The DP encoder effectively integrates LDM prior knowledge with image features.
• The LF decoder generates high-quality text-erased results by fusing heterogeneous features.

Conclusions:

• DiffEraser offers a robust solution for text removal across diverse image types.
• The framework demonstrates superior generalization capabilities, particularly in cross-domain applications.
• The proposed methods advance the state-of-the-art in scene text removal.