Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same authorSame journal

Sound source localization for spatial mapping of surgical actions in dynamic scenes.

International journal of computer assisted radiology and surgery·2026
Same author

Erratum for: Associations of MRI-derived Paraspinal IMAT and LMM with Cardiometabolic Risk Factors: Results from a German Cohort.

Radiology·2026
Same author

ProtoFlow: interpretable and robust surgical workflow modeling with learned dynamic scene graph prototypes.

International journal of computer assisted radiology and surgery·2026
Same author

Toward comprehensive real-time scene understanding in ophthalmic surgery through multimodal image fusion.

International journal of computer assisted radiology and surgery·2026
Same author

DefSynUS: Real-time patient-specific intrahepatic vessel identification via deformation-aware CT-US domain adaptation.

International journal of computer assisted radiology and surgery·2026
Same author

Smartphone photogrammetry for rapid 3D surface modeling of head and neck specimens to support frozen section communication: A feasibility pilot study.

Oral oncology·2026

Related Experiment Video

Updated: Nov 8, 2025

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping
09:41

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping

Published on: April 21, 2023

1.9K

Towards markerless surgical tool and hand pose estimation.

Jonas Hein1,2, Matthias Seibold3,4, Federica Bogo5

  • 1Research in Orthopedic Computer Science, University Hospital Balgrist, University of Zurich, Balgrist CAMPUS, Zurich, Switzerland. heinj@student.ethz.ch.

International Journal of Computer Assisted Radiology and Surgery
|April 21, 2021
PubMed
Summary
This summary is machine-generated.

Researchers developed a new framework to generate realistic synthetic and real-world data for markerless hand and instrument pose estimation in computer-assisted surgery, improving surgical tool tracking.

Keywords:
Deep learningHand poseObject poseSingle-shot pose estimationSynthetic data generation

More Related Videos

Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine
07:05

Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine

Published on: October 27, 2016

9.4K
Author Spotlight: Deciphering Electrical Networks Behind Complex Brain Activities and Disorders
05:49

Author Spotlight: Deciphering Electrical Networks Behind Complex Brain Activities and Disorders

Published on: November 1, 2024

1.1K

Related Experiment Videos

Last Updated: Nov 8, 2025

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping
09:41

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping

Published on: April 21, 2023

1.9K
Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine
07:05

Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine

Published on: October 27, 2016

9.4K
Author Spotlight: Deciphering Electrical Networks Behind Complex Brain Activities and Disorders
05:49

Author Spotlight: Deciphering Electrical Networks Behind Complex Brain Activities and Disorders

Published on: November 1, 2024

1.1K

Area of Science:

  • Computer-assisted surgery
  • Medical imaging
  • Robotics

Background:

  • Tracking surgical tools and activity is crucial for computer-assisted surgery.
  • Markerless hand and instrument pose estimation is essential for realistic surgical scenarios.

Purpose of the Study:

  • To present a data generation framework, dataset, and baseline methods for markerless hand and instrument pose estimation.
  • To facilitate further research in realistic surgical scenarios.

Main Methods:

  • Developed a rendering pipeline for generating inexpensive, realistic synthetic data for model pretraining.
  • Proposed a pipeline for capturing and labeling real data with ground truth hand and object poses.
  • Presented three state-of-the-art RGB-based pose estimation baselines.

Main Results:

  • Evaluated three baseline models on the generated synthetic and real datasets.
  • The best baseline achieved an average tool 3D vertex error of 16.7 mm on synthetic data and 13.8 mm on real data.
  • Performance is comparable to state-of-the-art RGB-based hand/object pose estimation.

Conclusions:

  • The study proposes the first synthetic and real data generation pipelines for hand and object pose labels in open surgery.
  • Presented baseline models for RGB-based object and hand pose estimation.
  • The synthetic data generation pipeline can help overcome data limitations in surgery and be applied to other medical fields.