Neural Wave Propagation for Surgical Video Action Recognition: A New Dataset and Baseline

Area of Science:

• Computer Vision
• Medical Robotics
• Artificial Intelligence

Background:

• Surgical video action recognition (SVAR) is vital for AI-driven surgical robotics.
• Existing SVAR datasets are limited in scale, resolution, annotation consistency, and action coverage.
• Current models struggle with SVAR due to architectures that ignore high-frequency details and lack spatial bias, requiring extensive data.

Purpose of the Study:

• Introduce LapSurg-230K, a large-scale, high-resolution dataset for SVAR.
• Propose WaveR, an attention-free baseline model for SVAR.
• Address the data scarcity challenge in surgical AI by developing a data-efficient model.

Main Methods:

• Developed LapSurg-230K with 7,569 clips (230,246 frames) covering 11 actions across 9 surgery types.
• Proposed WaveR, utilizing physical wave propagation for adaptive spatial-temporal context aggregation.
• WaveR preserves high-frequency surgical cues and possesses an innate physical inductive bias.

Main Results:

• WaveR demonstrates robustness under extreme data scarcity (as low as 30% training data).
• Achieved state-of-the-art accuracy in both surgical video action recognition and phase recognition tasks.
• LapSurg-230K supports comprehensive evaluation in full and progressive data volume settings.

Conclusions:

• LapSurg-230K and WaveR offer a significant advancement for surgical video analysis.
• WaveR's data-efficient design overcomes limitations of traditional models in limited-data domains.
• The dataset and model pave the way for more advanced AI applications in surgery.