Enhancing human spatial awareness through augmented reality technologies

Augmented reality (AR) has emerged as a powerful tool for enhancing human spatial awareness by overlaying digital information onto the physical world. This paper presents a review of the methodologies that enable AR-based spatial perception, with a focus on challenging environments such as underwater and disaster scenarios. We review state-of-the-art deep learning approaches for 3D data interpretation and completion, including voxel-based, point-based, and view-based methods. As part of this review, we implement an AR-enabled spatial awareness system, where the investigated deep learning solutions can be tested directly. In our approach, a robotic arm with an ultrasound sensor performs 2D scans underwater, from which a 3D point cloud of the scene is reconstructed. Using the reviewed deep learning networks, the point cloud is segmented in order to identify objects of interest, and point cloud completion is performed to infer missing structure. We report experimental results from synthetic data and underwater scanning trials, demonstrating that the system can recover and augment unseen spatial information for the user. We discuss the outcomes, including segmentation accuracy and completeness of reconstructions, as well as challenges such as data scarcity, noise, and real-time constraints. The paper concludes that, when combined with robust sensing and 3D deep learning techniques, AR enhances human spatial awareness in environments where direct perception is limited. The need for more adequate metrics to describe point clouds and for more labeled sonar datasets is discussed.