Explainable VQA-based ladder safety monitoring for fall-risk prevention on construction sites

Falls from height remain a major source of injuries and fatalities on construction sites, and ladder-related incidents contribute substantially to this burden. While object detection has been widely used in computer vision-based ladder safety research to identify equipment or unsafe postures, it is often insufficient for assessing compliance with ladder safety regulations that depend on contextual conditions, working height, and relationships among multiple objects and workers. This study proposes a regulation-guided Visual Question Answering (VQA) framework to support automated ladder safety compliance assessment for fall-risk prevention in construction.

Ladder safety regulations were encoded into nine structured questions representing major compliance factors, including ladder type, working height, top-step usage, outrigger installation, personal protective equipment (PPE), and two-person team requirements. A domain-specific dataset of 11,157 ladder-related images was compiled for training and evaluation. The BLIP-1 model was fine-tuned to interpret ladder work scenes and produce sentence-level answers to regulation-oriented questions. These responses were integrated into an automatic safety reporting module that generated both compliance decisions and explanatory descriptions.

The proposed framework achieved an overall sentence-level accuracy of 73.87% based on semantic similarity evaluation across 11,145 responses. For final safety judgment, it achieved F1-score of 0.757.

The results indicate that regulation-aware VQA can extend ladder safety monitoring beyond conventional object-detection-based approaches by enabling explainable and context-aware evaluation of rule compliance. The framework demonstrates the feasibility of integrating engineering safety rules with visual scene understanding for automated assessment of ladder work conditions.

The proposed method can assist site safety personnel by automatically identifying potential ladder safety violations and generating interpretable text-based reports for inspection and decision support. Such a system may improve the consistency of ladder safety checks and support future AI-enabled fall prevention practices in construction environments.