Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Photoacoustic device fingerprints induce bias in deep learning models.

Scientific reports·2026
Same author

Current validation practice undermines surgical AI development.

ArXiv·2026
Same author

Erratum: Learning to simulate realistic human diffuse reflectance spectra (Erratum).

Journal of biomedical optics·2026
Same author

Correction: A complete benchmark for polyp detection, segmentation and classification in colonoscopy images.

Frontiers in oncology·2026
Same author

Learning to simulate realistic human diffuse reflectance spectra.

Journal of biomedical optics·2026
Same author

Psychosocial Support Needs and Utilization of Support Among Survivors of Cancer in Young Adulthood in Switzerland: A Report From the Adolescent and Young Adult (AYA) Psychosocial Health Study.

Psycho-oncology·2026
Same journal

Correspondence-free local-to-global liver deformation correction via implicit neural representation and biomechanical model.

International journal of computer assisted radiology and surgery·2026
Same journal

BronchoLumen: analysis of recent YOLO-based architectures for real-time bronchial orifice detection in video bronchoscopy.

International journal of computer assisted radiology and surgery·2026
Same journal

Model-guided medicine for early diagnosis of transthyretin-associated cardiac amyloidosis using multimodal data integration and standardized interoperable models (the CRONOS-ATTR study).

International journal of computer assisted radiology and surgery·2026
Same journal

Electromagnetic navigation for femoral osteotomy using high-accuracy X-ray-to-CT registration.

International journal of computer assisted radiology and surgery·2026
Same journal

Modular instrument actuation unit for robotic-assisted systems in laparoscopic surgery.

International journal of computer assisted radiology and surgery·2026
Same journal

Pose-aware deep perceptual similarity for iterative 2D/3D registration of knee joints using contrastive learning.

International journal of computer assisted radiology and surgery·2026
See all related articles

Related Experiment Video

Updated: Sep 21, 2025

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility
07:46

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility

Published on: August 9, 2024

861

Robust hand tracking for surgical telestration.

Lucas-Raphael Müller1,2, Jens Petersen3, Amine Yamlahi4

  • 1Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany. lucas-raphael.mueller@dkfz.de.

International Journal of Computer Assisted Radiology and Surgery
|May 27, 2022
PubMed
Summary
This summary is machine-generated.

This study introduces an augmented reality (AR) hand tracking system for surgical telestration, improving trainee mentorship. The system demonstrates high accuracy and speed, paving the way for enhanced surgical education.

Keywords:
Computer visionDeep learningHand trackingSurgical data scienceTelestration

More Related Videos

Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation
10:23

Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation

Published on: September 2, 2025

59
Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery
05:12

Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery

Published on: August 12, 2021

2.2K

Related Experiment Videos

Last Updated: Sep 21, 2025

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility
07:46

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility

Published on: August 9, 2024

861
Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation
10:23

Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation

Published on: September 2, 2025

59
Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery
05:12

Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery

Published on: August 12, 2021

2.2K

Area of Science:

  • Medical Education Technology
  • Computer Vision in Surgery
  • Augmented Reality Applications

Background:

  • Human error is a significant cause of post-operative mortality, highlighting the critical need for effective surgical training.
  • Surgical telestration offers a method for experienced surgeons to mentor trainees remotely, enhancing skill transfer.
  • Existing telestration methods often rely on 2D video overlays, limiting intuitive interaction.

Purpose of the Study:

  • To develop and validate a real-time hand tracking pipeline for augmented reality (AR) surgical telestration.
  • To enable intuitive mentorship by visualizing surgical hands directly in the trainee's field of view.
  • To move beyond 2D overlays towards immersive AR visualization for surgical guidance.

Main Methods:

  • A three-module pipeline was designed for real-time hand tracking: coarse localization, hand segmentation for AR visualization, and keypoint regression for skeletal representation.
  • The system utilizes a neural network-based approach for robust hand tracking.
  • Semantic representation of hand motions was incorporated for structured reporting in teaching.

Main Results:

  • The algorithm achieved real-time performance with high accuracy across diverse conditions, validated on over 14,000 images.
  • Retrospective validation showed a 98% mean detection accuracy, 10.0 px keypoint regression accuracy, and a 0.95 Dice Similarity Coefficient.
  • Prospective validation confirmed uncompromised performance across varying sensors, operators, and gestures.

Conclusions:

  • The developed neural network-based hand tracking system is highly accurate and fast, making it suitable for AR surgical telestration.
  • This approach has the potential to significantly enhance surgical training and mentorship.
  • Further research is needed to evaluate the clinical utility and impact of this AR telestration system.