SGNet: Spectral-geometric neural network for structured representation learning

Convolutional neural networks (CNNs) and vision transformers (ViTs) represent two dominant paradigms in deep representation learning. However, CNNs are inherently limited by fixed local receptive fields that lack global shape awareness, while ViTs discard geometric structure by treating inputs as unordered token sets. To bridge this gap, we propose SGNet-a novel neural architecture that operates in a joint spectral-geometric representation space, where features are learned simultaneously in the local spatial domain and a data-adaptive spectral basis derived from the intrinsic geometry of the input. At the core of SGNet lies the Spectral-Geometric Operator (SGO), a differentiable layer that (1) constructs a geometric affinity graph from local feature structure, (2) implicitly learns a low-rank spectral basis via approximation of the graph Laplacian's dominant eigenmodes conditioned on input content, and (3) fuses local geometric responses with global spectral coefficients through a learnable gating mechanism. Unlike classical spectral methods that rely on fixed bases (e.g., Fourier or graph Laplacian eigenvectors of a static domain), SGNet's basis is end-to-end learnable and input-dependent, enabling shape-aware frequency reasoning without handcrafted transforms or quadratic-complexity attention. We provide theoretical justification for the stability and expressivity of SGO under geometric perturbations. Empirically, SGNet achieves state-of-the-art performance on image classification (CIFAR-100, ImageNet-1 K) and 3D object recognition (ModelNet40), outperforming CNNs, ViTs, and modern hybrids with fewer parameters and FLOPs. Our work establishes a new direction in neural representation learning: adaptive operator design over data-induced manifolds. The source code of the propose SGNet is publicly available at github.com/livingjesus/SGNet.