Differentiable Visual Computing: Challenges and Opportunities.

Area of Science:

• Computer Vision
• Machine Learning
• Scientific Computing

Background:

• Classical algorithms in visual computing possess valuable domain-specific insights, leading to robust, interpretable, and efficient models.
• Deep learning models require extensive data and gradient-based optimization to learn domain-specific insights from scratch.
• A gap exists in integrating the strengths of classical algorithms with modern data-driven deep learning methods.

Purpose of the Study:

• To explore the benefits and challenges of making classical visual computing algorithms differentiable.
• To enable gradient-based optimization for classical algorithms, combining their interpretability with deep learning's learning capabilities.
• To present novel approaches, as detailed in the author's thesis, for addressing the challenges in differentiating classical algorithms.

Main Methods:

• Investigating the differentiability of classical visual computing algorithms.
• Analyzing the challenges posed by discontinuities and irregular computation patterns in classical algorithms.
• Developing and evaluating techniques to integrate classical algorithms into gradient-based optimization frameworks.

Main Results:

• Demonstrated the feasibility of making certain classical visual computing algorithms differentiable.
• Identified key challenges, including discontinuities and irregular computation patterns, in applying gradient-based optimization to classical algorithms.
• The author's thesis represents an early contribution towards overcoming these integration challenges.

Conclusions:

• Combining classical algorithms with deep learning offers a promising direction for more robust and interpretable AI.
• Differentiability of classical algorithms is crucial for leveraging gradient-based optimization, but presents significant technical hurdles.
• Further research is needed to fully realize the potential of hybrid approaches in visual computing.