Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

"Deep-Onto" network for surgical workflow and context recognition.

Hirenkumar Nakawala1, Roberto Bianchi2, Laura Erica Pescatori3

  • 1Department of Electronics, Information and Bioengineering (DEIB), Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133, Milan, Italy. hirenkumar.nakawala@polimi.it.

International Journal of Computer Assisted Radiology and Surgery
|November 17, 2018
PubMed
Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

High-intensity focused ultrasound (HIFU) modeling: in vitro validation and integration into patient-specific planning tool.

Scientific reports·2026
Same author

SurgViVQA: temporally grounded video question answering for surgical scene understanding.

International journal of computer assisted radiology and surgery·2026
Same author

LumenGSLAM: online physically based rendering with Gaussian Splatting for robust endoscopic reconstruction and tracking.

International journal of computer assisted radiology and surgery·2026
Same author

Current validation practice undermines surgical AI development.

ArXiv·2026
Same author

Minds and machines: AI's transformative role in human identity and medicine.

Digital health·2026
Same author

Enhancing accuracy and explainability in colorectal lesion classification with attention-supervised Vision Transformers.

Computer methods and programs in biomedicine·2026
This summary is machine-generated.

This study introduces a hybrid deep learning and knowledge representation system for recognizing surgical workflow, improving context-aware decision-making in procedures like robot-assisted partial nephrectomy (RAPN). The system accurately identifies surgical steps and contextual elements, enhancing surgical planning and outcomes.

Area of Science:

  • Computer-assisted surgery
  • Artificial Intelligence in Medicine
  • Surgical Workflow Analysis

Background:

  • Current computer-assisted surgical systems primarily focus on phase recognition, neglecting crucial workflow sequences and contextual elements like instruments.
  • Effective surgical workflow recognition and context-aware systems are vital for improved decision-making, surgical planning, and enhanced patient outcomes.

Purpose of the Study:

  • To propose and evaluate a hybrid approach combining deep learning and knowledge representation for multi-level surgical workflow recognition.
  • To develop a system capable of recognizing not only surgical steps but also associated contextual elements such as actions, phases, and instruments.

Main Methods:

  • Implementation of the "Deep-Onto" network, an ensemble of deep learning models and knowledge management tools (ontology, production rules).
Keywords:
Deep learningKnowledge representationRobot-assisted partial nephrectomySurgical workflow

Related Experiment Videos

  • Utilized robot-assisted partial nephrectomy (RAPN) as a prototypical scenario, annotating videos with surgical entities like 'Step'.
  • Conducted experiments to recognize surgical steps, subsequent steps, and other surgical contexts, including inter-subject variability analysis.

    • Main Results:

    • The developed system achieved a prevalence-weighted macro-average (PWMA) recall of 0.83, PWMA precision of 0.74, and PWMA F1 score of 0.76 for recognizing 10 RAPN steps.
    • The system demonstrated an overall accuracy of 74.29% on 9 robot-assisted partial nephrectomy videos.
    • Successfully recognized surgical steps along with contextual information like 'Actions,' 'Phase,' and 'Instruments'.

    Conclusions:

    • The integration of deep learning and knowledge representation techniques offers a promising avenue for multi-level surgical workflow recognition.
    • The "Deep-Onto" network effectively facilitates the recognition of complex surgical workflows, paving the way for more advanced context-aware surgical systems.