Real-Time CNN Training and Compression for Neural-Enhanced Adaptive Live Streaming.

Area of Science:

• Computer Science
• Artificial Intelligence
• Video Streaming

Background:

• Live video streaming quality degrades significantly in poor network conditions.
• Existing methods struggle to provide high-resolution video efficiently under network constraints.

Purpose of the Study:

• To propose a real-time convolutional neural network (CNN) training and compression method for high-quality live video delivery.
• To enhance the user experience in live streaming, especially in adverse network environments.

Main Methods:

• Server delivers low-resolution video segments with a corresponding CNN for super-resolution (SR).
• Client applies the SR CNN to recover high-resolution frames.
• Real-time CNN training employs curriculum-based learning and promotes overfitting for rapid accuracy.
• Transfers only quantized residual values of CNN parameters to minimize data transmission.

Main Results:

• The proposed neural-enhanced adaptive live streaming pipeline (NEALS) achieves higher SR accuracy.
• NEALS demonstrates a lower CNN compression loss rate within constrained training times.
• Achieves 15% to 48% higher quality of user experience compared to state-of-the-art systems.

Conclusions:

• The NEALS method effectively improves live video quality in poor network conditions.
• Real-time CNN training and efficient compression are key to enhanced live streaming.
• This approach offers a significant improvement in user experience for live video services.