Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

ESD-VesNet: uncertainty-aware vessel segmentation network for endoscopic submucosal dissection with hard negative mining.

International journal of computer assisted radiology and surgery·2026
Same author

Transferable Deep Reinforcement Learning With Edge-Contour-Depth Fusion for Autonomous Wireless Capsule Endoscopy Navigation.

Advanced science (Weinheim, Baden-Wurttemberg, Germany)·2026
Same author

How can reasoning capability empower the AI copilot robot in endoscopic surgery.

NPJ digital medicine·2026
Same author

Mapping the facial nerve beyond the skull base: A comparative study of DESS and SPACE MRI sequences.

AJNR. American journal of neuroradiology·2026
Same author

Current validation practice undermines surgical AI development.

ArXiv·2026
Same author

A Real-time Scale-robust Network for Glottis Segmentation in Nasal Transnasal Intubation.

IEEE journal of biomedical and health informatics·2026
Same journal

Co-assistant networks by pathology foundation model and convolutional neural network for gigapixel whole slide image analysis.

Medical image analysis·2026
Same journal

MBAS2024: A large-scale benchmark for multi-class bi-atrial segmentation in multi-center contrast-enhanced MRIs.

Medical image analysis·2026
Same journal

Respiratory motion augmentation for personalized super-resolution (RMApSR) of 3D cine MR images in MRI-guided radiotherapy.

Medical image analysis·2026
Same journal

Biom3d, a modular framework to host and develop 3D segmentation methods.

Medical image analysis·2026
Same journal

Embracing intra-class heterogeneity for semi-supervised medical image segmentation: From diversity to precision.

Medical image analysis·2026
Same journal

Real-time patient-specific microwave ablation zone prediction via a unified bioheat solver and MRI-informed perturbation learning.

Medical image analysis·2026
See all related articles

Related Experiment Video

Updated: Dec 3, 2025

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility
07:46

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility

Published on: August 9, 2024

1.1K

ST-MTL: Spatio-Temporal multitask learning model to predict scanpath while tracking instruments in robotic surgery.

Mobarakol Islam1, Vibashan Vs2, Chwee Ming Lim3

  • 1BME Dept, National University of Singapore (NUS).

Medical Image Analysis
|October 31, 2020
PubMed
Summary
This summary is machine-generated.

We developed a Spatio-Temporal Multi-Task Learning (ST-MTL) model for robotic surgery. This model automates camera control through instrument tracking and saliency detection, improving surgical efficiency.

Keywords:
Multitask learningOptimizationRobot-assisted surgerySurgical endoscope guidanceSurgical instrument segmentationVisual attention

More Related Videos

Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation
10:25

Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation

Published on: September 2, 2025

338
Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery
05:12

Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery

Published on: August 12, 2021

2.3K

Related Experiment Videos

Last Updated: Dec 3, 2025

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility
07:46

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility

Published on: August 9, 2024

1.1K
Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation
10:25

Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation

Published on: September 2, 2025

338
Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery
05:12

Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery

Published on: August 12, 2021

2.3K

Area of Science:

  • Robotics
  • Computer Vision
  • Medical Imaging

Background:

  • Image-guided robotic surgery requires precise instrument tracking and camera control.
  • Automating camera control can reduce surgeon cognitive load and improve operational efficiency.
  • Current methods struggle with optimizing multiple tasks within multi-task learning frameworks.

Purpose of the Study:

  • To propose an end-to-end trainable Spatio-Temporal Multi-Task Learning (ST-MTL) model for real-time surgical instrument segmentation and task-oriented saliency detection.
  • To address the challenge of optimizing multiple loss functions in shared-parameter multi-task learning models.
  • To enhance camera control automation for improved surgical outcomes.

Main Methods:

  • Developed a Spatio-Temporal Multi-Task Learning (ST-MTL) model with a shared encoder and spatio-temporal decoders.
  • Introduced an asynchronous spatio-temporal optimization (ASTO) technique for independent gradient calculation.
  • Designed a competitive squeeze and excitation unit for dynamic feature recalibration.
  • Enhanced the long-short term memory (LSTM) module for improved spatio-temporal dependency capture.
  • Incorporated Sinkhorn regularized loss for efficient task-oriented saliency detection.

Main Results:

  • The ST-MTL model achieved superior performance in surgical instrument segmentation and task-oriented saliency detection.
  • Generated task-aware saliency maps and instrument scanpaths on the MICCAI 2017 dataset.
  • Outperformed state-of-the-art methods across multiple evaluation metrics.
  • Demonstrated outstanding performance in the robotic instrument segmentation challenge.

Conclusions:

  • The proposed ST-MTL model with ASTO effectively integrates instrument segmentation and saliency detection for enhanced robotic surgery.
  • The model's ability to automate camera control by learning task-oriented attention shows significant potential for reducing operation time.
  • This approach facilitates surgery for both surgeons and patients by allowing greater focus on instrument manipulation.