Tabular context-aware optical character recognition and tabular data reconstruction for historical records.

Area of Science:

• Computer Science
• Digital Humanities
• Information Science

Background:

• Digitizing historical tabular records is crucial for data preservation and analysis.
• Challenges include complex layouts, mixed text types, and degraded document quality.
• Existing methods struggle with the intricacies of historical documents.

Purpose of the Study:

• To present a comprehensive framework for robust historical table digitization.
• To introduce a novel dataset and context-aware text extraction approach.
• To develop an enhanced end-to-end OCR pipeline for improved accuracy.

Main Methods:

• Developed UoS_Data_Rescue, a dataset of 1,113 historical logbooks with over 594,000 annotated text cells.
• Proposed TrOCR-ctx, a context-aware text extraction approach to minimize cascading errors.
• Integrated TrOCR-ctx with ByT5 in a unified OCR and post-OCR correction framework.

Main Results:

• Achieved a word error rate of 0.049 and a character error rate of 0.035.
• Outperformed existing methods by up to 41% in OCR tasks.
• Demonstrated a 10.74% improvement in table reconstruction tasks.

Conclusions:

• The proposed framework offers a robust solution for large-scale digitization of complex tabular documents.
• The system enhances recognition accuracy for multilingual and degraded text.
• The dataset and implementation are available as open-source resources for broader application.