SignMamba: Sparse spatial-temporal state-space modeling for continuous sign language recognition

Sign language exhibits significant semantic sparsity across both spatial and temporal dimensions. Spatially, task-relevant motion is often concentrated in regions such as the hands and face, while the expansive static background is generally less informative. Temporally, the gesture stream is interspersed with considerable redundant information, such as pauses or slow transitions, meaning key semantics are condensed into a few sparse moments on the temporal axis. However, existing methods typically employ a dense modeling paradigm, failing to fully exploit this inherent property. To this end, we propose SignMamba, a novel framework that efficiently models global context through dual sparse sampling. To address spatial sparsity, we design a Mamba-based motion-aware selective scanning strategy. This strategy adaptively focuses on high-motion regions like the hands and face, thereby filtering out background noise and precisely modeling key actions. To tackle temporal sparsity, we further introduce a temporal sparse aggregation strategy. This method first adaptively selects the most salient spatial information between adjacent frames to capture local dynamic changes. Subsequently, through keyframe sampling, it performs global temporal modeling exclusively on the most information-rich frames. This allows the state-space model to bypass redundant segments and effectively capture long-range semantic dependencies. Experiments on public datasets, including PHOENIX14, PHOENIX14-T, and CSL-Daily, demonstrate that SignMamba achieves competitive performance, supporting the effectiveness of modeling based on semantic sparsity.