SPARTAN: automated table detection and extraction from documents using advanced OpenCV heuristics and OCR techniques

Area of Science:

• Computer Science
• Artificial Intelligence
• Document Analysis

Background:

• The proliferation of digital documents necessitates efficient tabular data extraction.
• Current deep learning methods offer high accuracy but are computationally expensive and lack transparency.
• There is a need for resource-efficient, explainable solutions for large-scale PDF table extraction.

Purpose of the Study:

• To introduce SPARTAN, an open-source, heuristic-based pipeline for high-fidelity table detection and extraction from PDFs.
• To demonstrate SPARTAN's effectiveness without requiring model training or GPUs.
• To provide a practical and cost-efficient alternative to complex deep learning models.

Main Methods:

• SPARTAN utilizes lightweight OpenCV image processing modules for table structure analysis.
• It incorporates column whitespace analysis, region segmentation, and cell parsing.
• A modular OCR layer and optional LLM-driven post-processing enhance its capabilities.

Main Results:

• SPARTAN achieved 0.94 precision, 0.91 recall, and 0.93 F1-score on over 20,000 pages.
• It processed pages at 2.8 seconds per page on average CPU time with 96.7% OCR accuracy.
• The pipeline demonstrated superior accuracy and speed compared to existing methods like Tabula and Deepdoctection.

Conclusions:

• Modernized heuristics combined with high-quality OCR can rival or surpass deep learning performance.
• SPARTAN offers a cost-efficient, explainable, and deployable solution for PDF table extraction.
• The tool is suitable for small and medium enterprises, bridging the gap between rule-based systems and AI.