An Introduction to 3-D Sparse CNNs

Three-dimensional data are now central to computer vision, robotics, autonomous driving, medical imaging, augmented reality, and computer graphics. Yet, 3-D deep learning is difficult because geometric data are often large, irregular, and mostly empty. Dense voxel convolutional neural networks (CNNs) provide a simple extension of image CNNs to 3-D, but their memory and computation grow cubically with resolution. Sparse 3-D CNNs address this problem by storing and processing only the active parts of space, such as surface voxels or occupied cells in a point cloud. This article gives an accessible introduction to voxel-based CNNs, sparse convolution, and the data structures that make sparse 3-D networks practical, with special attention to octrees, hash tables, and the practical lessons and pitfalls that matter in real systems.