A novel approach to FRUC using discriminant saliency and frame segmentation

Area of Science:

• Computer Vision
• Image Processing
• Video Enhancement

Background:

• Motion-compensated frame interpolation (MCFI) increases video frame rates but struggles with motion estimation errors in occluded or repetitive areas.
• Accurate motion field estimation is crucial for high-quality video interpolation.

Purpose of the Study:

• To propose an algorithm for refining motion vector fields to improve both objective and subjective quality of MCFI.
• To address limitations of current MCFI techniques in handling occlusions and repetitive structures.

Main Methods:

• Utilized a discriminant saliency classifier to identify critical motion field regions for human perception.
• Applied multistage motion vector refinement (MVR) to important regions based on local neighborhood likelihood.
• Employed frame segmentation with normalized cuts for homogeneous regions below the saliency threshold to ensure consistent motion.

Main Results:

• Experimental results show significant improvements over existing frame rate up-conversion (FRUC) methods.
• The proposed algorithm enhances both objective metrics and subjective visual quality of interpolated videos.
• Demonstrated effective refinement of motion vectors in challenging video regions.

Conclusions:

• The developed algorithm effectively refines motion vector fields for superior MCFI.
• This approach offers a robust solution for improving video interpolation quality, particularly in complex scenes.
• The method enhances visual fidelity in frame rate up-conversion applications.