Enabling scale and rotation invariance in convolutional neural networks with retina like transformation

Traditional convolutional neural networks (CNNs) struggle with scale and rotation transformations, resulting in reduced performance on transformed images. Previous research focused on designing specific CNN modules to extract transformation-invariant features. However, these methods lack versatility and are not adaptable to a wide range of scenarios. Drawing inspiration from human visual invariance, we propose a novel brain-inspired approach to tackle the invariance problem in CNNs. If we consider a CNN as the visual cortex, we have the potential to design an "eye" that exhibits transformation invariance, allowing CNNs to perceive the world consistently. Therefore, we propose a retina module and then integrate it into CNNs to create transformation-invariant CNNs (TICNN), achieving scale and rotation invariance. The retina module comprises a retina-like transformation and a transformation-aware neural network (TANN). The retina-like transformation supports flexible image transformations, while the TANN regulates these transformations for scaling and rotation. Specifically, we propose a reference-based training method (RBTM) where the retina module learns to align input images with a reference scale and rotation, thereby achieving invariance. Furthermore, we provide mathematical substantiation for the retina module to confirm its feasibility. Experimental results also demonstrate that our method outperforms existing methods in recognizing images with scale and rotation variations. The code will be released at https://github.com/JiaHongZ/TICNN.