Enhancing enterprise decision support via swin transformer-based OCR-free information extraction

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Information extraction is vital for knowledge graphs, intelligence analysis, and multimodal retrieval.
• End-to-end OCR-free methods face challenges in balancing fine-grained character detail extraction with complex layout modeling.

Purpose of the Study:

• To propose a novel hybrid encoder architecture for OCR-free information extraction.
• To address the limitations of existing methods in handling both character details and global layouts.

Main Methods:

• Developed a hybrid encoder combining Convolutional Neural Networks (CNNs) and Swin Transformers.
• Introduced a geometry-aware asymmetric downsampling strategy using ConvNext (CN) and Swin-T modules.
• CN module compresses height for character distinction; Swin-T module reduces width for long-range dependencies.

Main Results:

• The proposed method achieved superior information extraction accuracy on CORD and IIT-CDIP datasets.
• Outperformed existing OCR-free end-to-end information extraction techniques.
• Demonstrated potential for advancing intelligent operations and maintenance.

Conclusions:

• The hybrid encoder effectively balances fine-grained character extraction and global layout modeling.
• The geometry-aware asymmetric downsampling strategy is key to the method's success.
• The approach shows promise for improving information extraction in complex document analysis.