Deep Loss Convexification for Learning Iterative Models.

Area of Science:

• Computer Vision
• Machine Learning
• Optimization

Background:

• Iterative methods for tasks like 3D point cloud registration often face challenges with local optimality due to non-convex optimization.
• Existing deep learning approaches can still be susceptible to saddle points and suboptimal solutions.

Purpose of the Study:

• To develop a novel method, Deep Loss Convexification (DLC), that reshapes the loss landscape of deep iterative methods.
• To ensure locally convex-like loss landscapes around ground truth predictions, mitigating local optima.

Main Methods:

• DLC utilizes over-parameterized neural networks to learn a desired loss landscape shape.
• Adversarial training manipulates ground-truth predictions, not input data, to achieve this reshaping.
• Star-convexity is employed as a geometric constraint, introducing novel hinge losses.

Main Results:

• The proposed DLC method successfully reshapes loss landscapes into a more convex-like structure.
• This leads to near-optimal predictions and improved performance in tested applications.
• State-of-the-art results were achieved on recurrent neural network training, 3D point cloud registration, and multimodal image alignment.

Conclusions:

• Deep Loss Convexification (DLC) offers a robust solution to the local optimality problem in deep iterative methods.
• The technique demonstrates broad applicability and effectiveness across various complex computer vision and machine learning tasks.
• DLC represents a significant advancement in optimizing deep learning models for registration and alignment problems.