RT-GAN: Recurrent Temporal GAN for Adding Lightweight Temporal Consistency to Frame-Based Domain Translation...

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Colonoscopy videos are rarely saved due to large file sizes, limiting AI model training data.
• Current AI models for colonoscopy are often trained on individual frames, lacking temporal consistency.
• Training temporally-consistent AI models requires substantial computational and memory resources.

Purpose of the Study:

• To present a lightweight solution, Recurrent Temporal GAN (RT-GAN), for incorporating temporal consistency into colonoscopy AI models.
• To reduce the computational and memory requirements for training temporally-consistent deep learning models.
• To demonstrate the effectiveness of RT-GAN on key colonoscopy tasks and release a novel temporal dataset.

Main Methods:

• Developed RT-GAN, a Recurrent Temporal Generative Adversarial Network with a tunable temporal parameter.
• Applied RT-GAN to individual frame-based AI approaches to enhance temporal consistency.
• Evaluated RT-GAN on haustral fold segmentation and realistic colonoscopy video generation.

Main Results:

• RT-GAN reduces training requirements by a factor of 5 compared to traditional methods.
• Demonstrated effectiveness in haustral fold segmentation, crucial for identifying missed surfaces.
• Successfully generated realistic colonoscopy simulator videos, aiding in training and development.

Conclusions:

• RT-GAN offers an efficient method for achieving temporal consistency in colonoscopy AI, significantly lowering training costs.
• The developed temporal dataset and RT-GAN provide valuable resources for advancing AI in colonoscopy.
• This approach facilitates the development of more robust and reliable AI tools for colonoscopy analysis.