Unsupervised Global and Local Homography Estimation with Coplanarity-Aware GAN

Area of Science:

• Computer Vision
• Machine Learning
• Geometric Deep Learning

Background:

• Unsupervised homography learning offers label-free training but struggles with multi-plane scenes due to unaddressed parallax.
• Existing methods often fail to accurately predict homography when multiple planes are present, leading to compromised alignment.

Purpose of the Study:

• To develop a novel unsupervised method for accurate homography estimation that explicitly handles plane-induced parallax.
• To improve alignment in scenes with multiple depth planes by extending the homography estimation framework.

Main Methods:

• Proposed HomoGAN utilizes a multi-scale transformer for coarse-to-fine homography prediction from image feature pyramids.
• An unsupervised Generative Adversarial Network (GAN) enforces coplanarity constraints via region masking and discriminator-based validation.
• Extended to MeshHomoGAN for local mesh-grid homography estimation, applying plane constraints to individual mesh cells.

Main Results:

• HomoGAN effectively guides unsupervised homography estimation to focus on the dominant plane.
• MeshHomoGAN demonstrates superior performance in aligning scenes with multiple depth planes.
• Achieved a 22% reduction in matching error compared to previous state-of-the-art (SOTA) methods.

Conclusions:

• The proposed HomoGAN and MeshHomoGAN methods significantly advance unsupervised homography estimation.
• Explicitly addressing plane-induced parallax and enabling local mesh-grid constraints are key to handling complex scenes.
• The approach offers a robust and accurate solution for challenging homography learning tasks.