REACT: Relation-Aware Class Activation-Guided Mixture-of-Experts for Spatio-Temporal Triplet Recognition

Video-based surgical action-triplet recognition represents each surgical action as an ⟨instrument, verb, target⟩ triplet, which is fundamental for understanding complex surgical scenes. However, this task remains challenging due to heterogeneous spatio-temporal characteristics and complex inter-component dependencies. To address these issues, we propose a Relation-Aware Class Activation-guided Mixture-of-Experts framework (REACT) for surgical triplet recognition. REACT jointly models spatial details and temporal dynamics while capturing inter-component relations via a Class Activation-guided Mixture-of-Experts (CAMoE) architecture. Class Activation Maps (CAMs) serve as weak semantic priors to guide expert routing, while Feature Modulation Experts (FMEs) adaptively align heterogeneous component features into a shared semantic space. Extensive experiments on CHOLECT45 and CHOLECTRIPLET show that REACT achieves state-of-the-art triplet-level performance with strong relational consistency. Additional evaluation on SIRNET dataset further demonstrates cross-procedure generalization. Code is available at https://github.com/accusemaker/REACT.