Keyframe extraction from laparoscopic videos based on visual saliency detection

Area of Science:

• Medical imaging
• Computer vision
• Surgical technology

Background:

• Laparoscopic surgery videos offer valuable data for technique evaluation, training, and documentation.
• Efficient representation of surgical video content requires extracting keyframes with semantic information.

Purpose of the Study:

• To present a novel method for keyframe extraction from individual shots of laparoscopic surgical videos.
• To improve the semantic information and temporal consistency of extracted keyframes.

Main Methods:

• Video shots segmented using an objectness model.
• Frame saliency maps (color, motion, texture) generated to model visual attention.
• Hidden Markov Model with Autoregressive process (HMMAR) used for temporal segmentation.
• Keyframes selected based on highest state-conditional probability.

Main Results:

• Tested on 168 laparoscopic cholecystectomy video shots.
• Proposed method achieved 81% color consistency with ground truth, outperforming a comparison method (77%).
• Demonstrated superior temporal consistency compared to other state-of-the-art methods.

Conclusions:

• The novel keyframe extraction method provides superior content and temporal consistency.
• Extracted keyframes offer rich semantic information for surgical video analysis, summarization, and retrieval.