Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

EndoLRMGS: Combining Large Reconstruction Modelling and Gaussian Splatting for Complete Endoscopic Scene Reconstruction.

IEEE transactions on medical imaging·2026
Same author

Robust quantification of ICG fluorescence perfusion in neonatal bowel surgery via deep point tracking.

International journal of computer assisted radiology and surgery·2026
Same author

Blob representation of robotic surgical scenes for position-aware video generation.

International journal of computer assisted radiology and surgery·2026
Same author

VerTE-MT: A Multi-Task Framework with Entropy-Guided Sampling for Vertebrae Segmentation and Localisation in CT.

IEEE journal of biomedical and health informatics·2026
Same author

AI based prediction of post prostatectomy urinary incontinence and its impact on quality of life: development and validation study.

Scientific reports·2026
Same author

SurgViVQA: temporally grounded video question answering for surgical scene understanding.

International journal of computer assisted radiology and surgery·2026

Related Experiment Video

Updated: Oct 4, 2025

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility
07:46

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility

Published on: August 9, 2024

875

Gesture Recognition in Robotic Surgery With Multimodal Attention.

Beatrice van Amsterdam, Isabel Funke, Eddie Edwards

    IEEE Transactions on Medical Imaging
    |February 2, 2022
    PubMed
    Summary

    This study fuses robot kinematic data and surgical video for better surgical gesture recognition. Integrating multimodal attention mechanisms improves accuracy and temporal structure in automated surgical analytics.

    More Related Videos

    Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation
    10:25

    Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation

    Published on: September 2, 2025

    103
    Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention
    06:37

    Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

    Published on: December 15, 2023

    4.3K

    Related Experiment Videos

    Last Updated: Oct 4, 2025

    Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility
    07:46

    Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility

    Published on: August 9, 2024

    875
    Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation
    10:25

    Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation

    Published on: September 2, 2025

    103
    Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention
    06:37

    Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

    Published on: December 15, 2023

    4.3K

    Area of Science:

    • Robotics
    • Computer Vision
    • Artificial Intelligence

    Background:

    • Automated surgical gesture recognition is crucial for analytics, skill assessment, and robotic automation.
    • Analyzing surgical motion patterns from robot kinematics alone is challenging due to instrument complexity and anatomical variability.
    • Surgical video offers contextual information but fusing it with kinematic data is difficult due to differing data characteristics.

    Purpose of the Study:

    • To develop an effective sensor fusion method for combining robot kinematics and surgical video data.
    • To improve the accuracy and temporal structure of surgical gesture recognition using multimodal data.
    • To dynamically weight kinematic and visual features for enhanced multimodal network training.

    Main Methods:

    • Integration of multimodal attention mechanisms within a two-stream temporal convolutional network.
    • Dynamic weighting of kinematic and visual feature representations based on computed relevance scores.
    • Evaluation on the JIGSAWS benchmark dataset and a novel in vivo dataset of robotic prostatectomy suturing segments.

    Main Results:

    • The proposed multimodal approach achieved higher accuracy and better temporal structure compared to unimodal solutions.
    • Attention scores provided interpretable insights into the network's understanding of individual sensor data.
    • The system demonstrated effective sensor fusion for surgical motion pattern analysis.

    Conclusions:

    • Multimodal attention mechanisms enable effective fusion of robot kinematics and surgical video for improved surgical gesture recognition.
    • The developed system offers a promising approach for advanced automated surgical analytics and skill assessment.
    • Dynamic weighting of sensor features enhances the robustness and interpretability of surgical data analysis.