OnUVS: An Online Motion Transfer Framework With Content-Texture Decoupling for High-Fidelity Ultrasound Video...

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Ultrasound (US) imaging is vital for diagnosing heart and pelvic conditions.
• Limited US video data for rare cases hinders sonographer training and AI model development.
• Synthesizing realistic US videos is challenging due to complex motion and image fidelity requirements.

Purpose of the Study:

• To develop a novel framework, OnUVS, for high-fidelity ultrasound video synthesis.
• To address the limitations of current methods in capturing anatomical motion and preserving image quality.
• To improve training data availability for sonographers and deep learning models.

Main Methods:

• Proposed OnUVS, an online feature-decoupling framework for US video synthesis.
• Incorporated keypoints via weakly supervised learning for enhanced motion representation.
• Utilized a dual decoder generator to balance content and texture, improving image fidelity.
• Employed a multi-scale discriminator for refining details and sharpness.
• Implemented an online learning strategy to ensure frame coherence via keypoint trajectory constraints.

Main Results:

• OnUVS demonstrated superior performance on echocardiographic and pelvic floor US datasets.
• Achieved a 22.08% improvement in motion consistency (Frame-by-Frame Video Difference - FVD).
• Achieved a 25.04% improvement in image fidelity (Fréchet Inception Distance - FID).

Conclusions:

• OnUVS effectively synthesizes high-fidelity ultrasound videos with realistic motion.
• The method overcomes key challenges in US video synthesis, enhancing motion and image quality.
• OnUVS provides a valuable tool for medical education and diagnostic AI development, with publicly available code.