Computer Vision for Autonomous Drill Jumbos: Detecting Non-Drillable Areas of a Mine Face

The mining industry is in need of automation due to increasing requirements like higher global demands for resources and deposits, which are deeper and more complex. Progressing underground mines lead to longer travel times to the mining face and thus a loss in productive working time, which has to be compensated by automation. Ultimately, stricter health and safety regulations and a decreasing number of skilled operators accelerate the need for automation further. Within the the drill-and-blast cycle in underground mining, the drilling of blast holes is a central step. While semi-automated and supporting systems exist that allow the automated execution of single process steps under supervision, to date, no system is available for the unsupervised blast hole drilling of a mine face. A precondition for unsupervised operation is a perception system, which allows independent decision-making of the machine. To address this gap, this work presents a novel vision system capable of segmenting a mine face into drillable and non-drillable areas, which can serve as the basis for the autonomous adaption of a drilling pattern. An area of the mine face is considered drillable if no leftover blast holes from the previous blast cycle are present and the surface angle is below a certain threshold. The system presented is based on a stereo camera setup mounted on a drill jumbo. The resulting 2D and 3D data are processed by software that employs AI-based computer vision techniques, as well as traditional algorithms. The system was validated, and the performance was verified in the K+S Zielitz mine. Experts assisted in the determination of operational parameters and empirically validated the system's performance. Additionally, the blast hole detection algorithm underwent a data-based analytical verification.