An explainable end-to-end computer vision pipeline for detection, segmentation, and reconstruction of occluded...

Images from crime scenes often show partially concealed weapons due to obstructions such as hands and clothing, as well as surveillance camera limitations, which affect the efficacy of traditional detection methods. This work proposes an explainable forensic pipeline for occluded weapons detection, segmentation, and reconstruction.

The proposed framework integrates RT-DETR-L, a transformer-based weapon detection model; MobileSAM for zero-shot segmentation of visible weapon regions; Stable Diffusion Inpainting for reconstruction of occluded regions; and a bilateral filter with Canny edge detection for forensic sketch generation. The detector was trained using synthetic occlusion augmentation on five weapon categories, such as firearm, grenade, knife, pistol, and rocket, and fire as an additional class for the environmental hazard indicator.

The RT-DETR-L model achieves a mAP@50 of 0.86 and a mAP@50-95 of 0.65, averaged over the five weapon classes, on a dataset with synthetic occlusion augmentation. Reconstruction quality meets minimum quality thresholds (SSIM > 0.54, PSNR > 16 dB, Region IoU > 0.80) up to approximately 50% occlusion. An interactive Streamlite application demonstrates the pipeline's feasibility in 42-46 seconds as a controlled laboratory prototype.

The proposed explainable forensic pipeline highlights the potential of combining transformer-based detection with generative AI to assist forensic weapon analysis under challenging occlusion scenarios. The generated reconstructions and sketches are intended as visual aids for expert review and must undergo independent forensic validation before any operational use.