Unified Architecture Adaptation for Compressed Domain Semantic Inference.

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Deep learning has advanced lossy image compression and semantic understanding separately.
• Existing methods for compressed domain inference are often architecture-specific or lack optimal accuracy.

Purpose of the Study:

• To develop a unified, lightweight solution for semantic inference directly from quantized latent features in the compressed domain.
• To enable semantic understanding without pixel reconstruction, bridging the gap between image compression and vision tasks.

Main Methods:

• Proposed a plug-and-play solution adaptable to popular learned image coders and deep vision models.
• Adapted pixel-domain neural models to process quantized latent features.
• Utilized knowledge transfer from pixel-domain counterparts for training compressed-domain models.

Main Results:

• Demonstrated compliance with popular learned image coders and vision models.
• Achieved over 3% higher top-1 accuracy in compressed domain classification compared to a baseline.
• Improved compressed domain semantic segmentation by over 7% mIoU at various data rates.

Conclusions:

• The proposed method offers a flexible and effective approach for compressed domain semantic inference.
• This work significantly enhances the performance of vision tasks on compressed image data.
• The findings pave the way for more integrated and efficient deep learning solutions in image compression and analysis.