Perceptual Video Coding for Machines via Satisfied Machine Ratio Modeling.

Area of Science:

• Computer Vision
• Machine Learning
• Signal Processing

Background:

• Current video compression for machines (VCM) methods inadequately address the diversity of machine perception.
• Existing VCM approaches fail to leverage machine-specific perceptual characteristics, limiting compression efficiency.

Purpose of the Study:

• To introduce Satisfied Machine Ratio (SMR) for statistically evaluating compressed visual data quality for machine analysis.
• To develop a robust VCM framework that accounts for a wider range of machine perceptual qualities.

Main Methods:

• Developed SMR by aggregating machine satisfaction scores based on perceptual differences.
• Created machine libraries and a large-scale SMR dataset for image classification and object detection.
• Proposed an SMR prediction model using deep feature differences and an auxiliary prediction task.

Main Results:

• SMR models demonstrated significant improvements in compression performance across various machines.
• The proposed models showed robust generalizability on unseen machines, codecs, datasets, and frame types.
• The auxiliary task enhanced the accuracy of SMR prediction.

Conclusions:

• SMR provides a statistically sound metric for evaluating machine perceptual quality in VCM.
• The developed SMR prediction models offer a promising direction for optimizing video compression for machine analysis.
• This work advances VCM by incorporating machine-specific perceptual factors for enhanced efficiency and generalizability.