Region-Based Static Video Stitching for Reduction of Parallax Distortion.

Area of Science:

• Computer Vision
• Image Processing
• Computational Photography

Background:

• Video stitching often suffers from parallax and misalignment, especially in complex scenes like sports stadiums with dynamic foregrounds.
• Existing methods struggle with plain textures or significant parallax, leading to degraded stitching quality.
• Semantic information has not been fully leveraged to address these challenges in video stitching.

Purpose of the Study:

• To propose a novel semantic segmentation-based static video stitching method.
• To reduce parallax and misalignment distortion in sports stadium scenes.
• To improve the overall quality and accuracy of stitched videos, particularly in challenging conditions.

Main Methods:

• Semantic segmentation is used to classify video frame segments.
• Region-based stitching is applied to segments of the same semantic class, assuming they lie on the same plane.
• Temporally consistent feature points are utilized for robust homography estimation, especially for plain or noisy segments.
• Stitched segments and foreground elements are synthesized based on area for the final frame.

Main Results:

• The proposed method significantly reduces parallax and misalignment distortion.
• Improvements are particularly notable in segments with plain textures or large parallax.
• Subjective quality, geometric distortion, and pixel distortion metrics show substantial enhancement compared to conventional methods.
• The method demonstrates superior performance in handling dynamic foreground objects and complex scene geometry.

Conclusions:

• Semantic segmentation provides a powerful approach to improve static video stitching.
• The region-based stitching strategy effectively handles scene geometry and object dynamics.
• This method offers a robust solution for high-quality video stitching in challenging environments like sports stadiums.