Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptation.

Area of Science:

• Computer Vision
• Image Processing

Background:

• Optical flow estimation excels in clean scenes but degrades under adverse weather.
• Existing domain adaptation methods struggle with the significant domain gap between clean and real-world degraded conditions.
• Adverse weather phenomena like fog and rain introduce distinct challenges to optical flow estimation.

Purpose of the Study:

• To develop a robust optical flow estimation method for real adverse weather conditions.
• To bridge the domain gap between clean and degraded visual data using a novel adaptation framework.
• To improve the transfer of motion knowledge from clean to real adverse weather domains.

Main Methods:

• Proposed a cumulative homogeneous-heterogeneous adaptation framework.
• Utilized synthetic degraded domains as an intermediate bridge for knowledge transfer.
• Leveraged depth-associated homogeneous features for static weather and heterogeneous features for dynamic weather.
• Employed cost volume correlation alignment for synthetic-to-real domain adaptation.
• Collected and annotated a real adverse weather optical flow dataset.

Main Results:

• The proposed framework enables progressive and explicit knowledge transfer from clean to real adverse weather.
• Demonstrated the effectiveness of using synthetic data as an intermediate domain.
• Showcased the ability to handle both static and dynamic adverse weather conditions.
• Achieved superior performance on the newly collected real adverse weather dataset.

Conclusions:

• The cumulative homogeneous-heterogeneous adaptation framework significantly enhances optical flow estimation in real adverse weather.
• The strategy of using synthetic data as an intermediate bridge is effective in closing the domain gap.
• The method provides a robust solution for challenging real-world scenarios previously unaddressed.