Related Experiment Videos
REACT: Relation-Aware Class Activation-Guided Mixture-of-Experts for Spatio-Temporal Triplet Recognition
IEEE Transactions on Medical Imaging
|July 22, 2026
Summary
This study introduces REACT, a novel framework for surgical triplet recognition. REACT effectively models complex surgical actions by capturing spatial, temporal, and relational dependencies, achieving state-of-the-art performance.
Area of Science:
- Computer Vision
- Medical Imaging
- Artificial Intelligence
Background:
- Surgical action triplet recognition is crucial for understanding surgical scenes.
- Existing methods struggle with heterogeneous spatio-temporal data and inter-component dependencies.
Purpose of the Study:
- To propose a novel framework, REACT, for accurate surgical triplet recognition.
- To jointly model spatial details, temporal dynamics, and inter-component relations.
Main Methods:
- Developed a Relation-Aware Class Activation-guided Mixture-of-Experts (REACT) framework.
- Utilized Class Activation Maps (CAMs) for expert routing and Feature Modulation Experts (FMEs) for feature alignment.
Main Results:
- Achieved state-of-the-art performance on CHOLECT45 and CHOLECTRIPLET datasets.
- Demonstrated strong relational consistency in surgical triplet recognition.
- Showcased cross-procedure generalization on the SIRNET dataset.
Conclusions:
- REACT effectively addresses challenges in surgical triplet recognition.
- The proposed method enhances understanding of complex surgical scenes.
- REACT offers a robust solution for video-based surgical action analysis.