Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Improving Translational Accuracy02:07

Improving Translational Accuracy

2.6K
2.6K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Corrigendum to "Visual modalities-based multimodal fusion for surgical phase recognition" [Comput. Biol. Med. 166 (2023) 107453].

Computers in biology and medicine·2026
Same author

Enhancing anatomical recognition by surgeons during pelvic lymph node dissection using artificial intelligence.

NPJ digital medicine·2026
Same author

Optical metasurfaces for general vision processing on the edge.

Nature·2026
Same author

Let us define arthroscopic surgeon's ergonomics! design of visualisation and evaluation approaches of surgical ergonomics.

International journal of computer assisted radiology and surgery·2026
Same author

Objective structured assessment of posterior capsular rupture management.

Graefe's archive for clinical and experimental ophthalmology = Albrecht von Graefes Archiv fur klinische und experimentelle Ophthalmologie·2026
Same author

Perceived Crises and Preparedness Gaps in Operating Room Nursing: A Qualitative Study of Training Priorities With Nurses and Educators.

Journal of nursing management·2026

Related Experiment Video

Updated: Aug 1, 2025

Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
08:15

Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision

Published on: March 28, 2025

674

PEg TRAnsfer Workflow recognition challenge report: Do multimodal data improve recognition?

Arnaud Huaulmé1, Kanako Harada2, Quang-Minh Nguyen1

  • 1Univ Rennes, INSERM, LTSI - UMR 1099, Rennes, F35000, France.

Computer Methods and Programs in Biomedicine
|April 29, 2023
PubMed
Summary

Combining video and kinematic data significantly improves surgical workflow recognition accuracy. However, the increased computational cost must be weighed against the marginal accuracy gains for real-time applications.

Keywords:
MultimodalOR of the futureSurgical process modelWorkflow recognition

More Related Videos

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
05:48

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Published on: August 9, 2024

1.6K
Profiling Maternal Behavior Responses During Whole-Brain Imaging
07:12

Profiling Maternal Behavior Responses During Whole-Brain Imaging

Published on: January 24, 2025

849

Related Experiment Videos

Last Updated: Aug 1, 2025

Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
08:15

Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision

Published on: March 28, 2025

674
Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
05:48

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Published on: August 9, 2024

1.6K
Profiling Maternal Behavior Responses During Whole-Brain Imaging
07:12

Profiling Maternal Behavior Responses During Whole-Brain Imaging

Published on: January 24, 2025

849

Area of Science:

  • Robotics in Medicine
  • Computer-Assisted Surgery
  • Surgical Workflow Analysis

Background:

  • Accurate, real-time surgical workflow recognition is crucial for context-aware computer-assisted surgery.
  • Surgical video has been the primary data source, but robot-assisted surgery enables new modalities like kinematics.
  • The added value of these new modalities in surgical workflow recognition has been underexplored.

Purpose of the Study:

  • To design and evaluate the "PEg TRAnsfer Workflow recognition" (PETRAW) challenge.
  • To develop surgical workflow recognition methods using single or multiple data modalities.
  • To assess the added value of different modalities for surgical workflow recognition.

Main Methods:

  • The PETRAW challenge utilized a dataset of 150 virtual peg transfer sequences.
  • Data included videos, kinematics, semantic segmentation, and annotations at phase, step, and activity levels.
  • Five tasks were proposed, focusing on single and multi-modality recognition, evaluated using application-dependent balanced accuracy (AD-Accuracy).

Main Results:

  • Seven teams participated, with four competing in all tasks.
  • The highest accuracy (90-93% AD-Accuracy) was achieved by combining video and kinematic data.
  • Multi-modality approaches significantly outperformed unimodal methods across all participating teams.

Conclusions:

  • Multi-modality approaches offer significant improvements in surgical workflow recognition accuracy.
  • The substantial increase in computation time for video/kinematic methods warrants careful consideration.
  • The PETRAW dataset is publicly available to foster further research in this field.