Unsupervised Multi-Latent Space RL Framework for Video Summarization in Ultrasound Imaging

Area of Science:

• Medical Imaging and Artificial Intelligence
• Ultrasound Technology
• Machine Learning for Healthcare

Background:

• The COVID-19 pandemic underscored the need for efficient tools in medical diagnostics.
• Current ultrasound video analysis often requires manual, time-consuming labeling.
• Rapid access to summarized clinical information is crucial for emergency departments and telemedicine.

Purpose of the Study:

• To develop an unsupervised reinforcement learning framework for summarizing ultrasound videos.
• To create a tool that avoids manual labeling for video summarization.
• To enhance triage capabilities in emergency settings and telemedicine through automated video analysis.

Main Methods:

• An unsupervised reinforcement learning framework with novel rewards was proposed.
• An attention ensemble of encoders projected high-dimensional image data into a low-dimensional latent space.
• A bi-directional long short-term memory network processed the latent space for video summarization.

Main Results:

• The framework achieved high agreement with ground truth on lung ultrasound (LUS) videos.
• Average precision exceeded 80%, with an average F1 score over 44 ±1.7 %.
• The approach reduced storage space by an average of 77%, benefiting telemedicine.

Conclusions:

• The proposed framework effectively summarizes ultrasound videos, providing classification labels and landmark segmentations.
• This tool can significantly improve triage efficiency in emergency departments and telemedicine applications.
• The AI-driven summarization reduces data storage and bandwidth requirements, facilitating wider adoption of telemedicine.