Temporally consistent sequence-to-sequence translation of cataract surgeries

Area of Science:

• Medical imaging
• Computer vision
• Surgical technology

Background:

• Image-to-image translation methods struggle with video data, causing artifacts in medical applications.
• Lack of diversity in cataract surgery datasets limits model generalizability.

Purpose of the Study:

• To develop spatio-temporal constraints for realistic video translation.
• To improve the temporal consistency of translated medical image sequences.

Main Methods:

• Introduced a motion-translation module for optical flow domain translation.
• Combined with a shared latent space model for enhanced image quality.
• Proposed novel quantitative metrics for temporal consistency evaluation.

Main Results:

• Achieved superior temporal consistency in translated sequences compared to baselines.
• Maintained competitive per-image translation quality.
• Demonstrated improved surgical phase prediction using consistently translated data.

Conclusions:

• The motion-translation module significantly enhances temporal consistency.
• Temporal constraints improve the utility of translated data for downstream tasks.
• Overcomes challenges in surgical data acquisition and annotation, boosting model performance.