Unified Video Reconstruction for Rolling Shutter and Global Shutter Cameras

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Video reconstruction (VR) methods are typically specialized for either global shutter or rolling shutter cameras.
• This fragmentation limits model generalization and hinders uniform VR development.

Purpose of the Study:

• To propose UniVR, a versatile framework for unified video reconstruction across diverse camera shutter types.
• To enable cross-shutter transfer learning and improve generalization in VR models.

Main Methods:

• Developed a parameter-free shutter adapter to encode various shutter types into a unified representation.
• Integrated the shutter adapter into existing VR architectures (SoftSplat, Super-SloMo, RIFE) to create shutter-generic models.
• Evaluated performance on novel shutters with and without fine-tuning.

Main Results:

• Pre-trained UniVR models achieved reasonable performance on unseen shutters without fine-tuning.
• Fine-tuned models surpassed shutter-specific methods, establishing new state-of-the-art results.
• Demonstrated strong generalization capabilities across different shutter types.

Conclusions:

• UniVR offers a unified approach to video reconstruction, overcoming the limitations of shutter-specific methods.
• The proposed framework significantly enhances the versatility and generalization of VR models.
• UniVR paves the way for more robust and adaptable video reconstruction solutions.