Related Experiment Video
Updated: Aug 3, 2025
Operation of the Collaborative Composite Manufacturing CCM System
Published on: October 1, 2019
Rethinking causality-driven robot tool segmentation with temporal constraints
Hao Ding1, Jie Ying Wu2, Zhaoshuo Li3
1Department of Computer Science, Johns Hopkins University, 3400 N. Charles St, Baltimore, MD, 21218, USA. hding15@jhu.edu.
Purpose:
Vision-based robot tool segmentation plays a fundamental role in surgical robots perception and downstream tasks. CaRTS, based on a complementary causal model, has shown promising performance in unseen counterfactual surgical environments in the presence of smoke, blood, etc. However, CaRTS requires over 30 iterations of optimization to converge for a single image due to limited observability.
Method:
To address the above limitations, we take temporal relation into consideration and propose a temporal causal model for robot tool segmentation on video sequences. We design an architecture named Temporally Constrained CaRTS (TC-CaRTS). TC-CaRTS has three novel modules to complement CaRTS-temporal optimization pipeline, kinematics correction network, and spatial-temporal regularization.
Results:
Experiment results show that TC-CaRTS requires fewer iterations to achieve the same or better performance as CaRTS on different domains. All three modules are proven to be effective.
Conclusion:
We propose TC-CaRTS, which takes advantage of temporal constraints as additional observability. We show that TC-CaRTS outperforms prior work in the robot tool segmentation task with improved convergence speed on test datasets from different domains.
