Hyper-Counter: Cross-Modal Hypergraph Fusion for RGB-T Crowd Counting

In RGB-T crowd counting, the performance of existing methods is often constrained by two major challenges. On the one hand, the inherent spatial misalignment between RGB and thermal infrared images makes cross-modal feature alignment and fusion extremely difficult. On the other hand, effectively integrating the complementary semantic information from both modalities and capturing their complex high-order associations remains a significant challenge. To address these issues, we propose Hyper-Counter, a novel framework based on crossmodal hypergraph fusion, introducing hypergraph learning into RGB-T crowd counting for the first time. Specifically, we utilize parallel Transformer backbones to extract multi-scale features and employ convolutional blocks to preliminarily fuse the features from both modalities into joint features. Subsequently, we design a cross-modal hypergraph perception module (CroM-HP), where feature patches are treated as vertices, and a cross-modal hypergraph is constructed based on the joint and cross-modal features. Hypergraph neural networks are then used to achieve efficient semantic information interaction and fusion. Through CroM-HP, the model can associate corresponding regions across modalities in a spatially flexible manner and effectively capture the intricate complementary semantic correlations. Moreover, a pyramid structure is adopted in the multi-level CroM-HPs to achieve cross-scale semantic transmission. Experiments on two widely used datasets, RGBT-CC and Drone-RGBT, demonstrate that our framework consistently outperforms existing RGBT crowd counting methods. In particular, compared with the representative prior method BMCC on Drone-RGBT, Hyper- Counter improves the GAME(0) and RMSE metrics by 7.58% and 9.71%, respectively.