Automated overview of complete endoscopies with unsupervised learned descriptors

Area of Science:

• Medical Imaging
• Computational Pathology
• Endoscopic Video Analysis

Background:

• Endoscopy procedures generate extensive video data, posing challenges for manual review.
• Efficient analysis of endoscopic videos is crucial for clinical decision-making and research.

Purpose of the Study:

• To automate the initial analysis of complete endoscopy videos, identifying relevant content.
• To reduce clinicians' review time and facilitate downstream tasks like video summarization and 3D reconstruction.

Main Methods:

• Extracting endoscopic video frame representations using a learned embedding model.
• Clustering descriptors to identify visual patterns and key scene types (e.g., surgery, clear visibility).
• Segmenting videos into informative and non-informative parts.

Main Results:

• Demonstrated good performance in identifying surgery segments and varying visibility conditions on colonoscopy videos.
• Generated structured overviews separating useful segments from irrelevant ones.
• Illustrated suitability as preprocessing for 3D reconstruction and video summarization.

Conclusions:

• Automated endoscopy overview generation helps clinicians focus on relevant content like good visibility sections and surgical actions.
• Facilitates faster recording review and effective video preprocessing for downstream tasks.