Automated hand-marked semantic text recognition from photographs.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Automated text recognition faces challenges with handwritten or marked text.
• Construction defect identification requires efficient processing of numerous tagged photographs.

Purpose of the Study:

• To develop and compare automated methods for recognizing hand-marked semantic text (HMSTR) on construction defect tags.
• To evaluate the effectiveness of scene text recognition (STR)-based, two-step HMSTR, and lumped approaches for this task.

Main Methods:

• Investigated three HMSTR approaches: modified STR, a two-step method (localization then classification), and a single-step lumped object detection method.
• Evaluated performance using F1 scores for recognizing circled and check-marked text on construction tags.

Main Results:

• The two-step HMSTR approach achieved the highest F1 score (0.92) for circled text recognition.
• The STR approach yielded an F1 score of 0.87, while the lumped approach scored 0.78.
• The two-step HMSTR approach demonstrated generalizability with an F1 score of 0.88 for check-marked text.

Conclusions:

• The two-step HMSTR approach is the most effective for automated recognition of hand-marked text on construction defect tags.
• This method shows potential for broader applications in recognizing marked text in documents and reports.
• Further research can extend these techniques to diverse datasets and real-world scenarios.