CRefNet: Learning Consistent Reflectance Estimation With a Decoder-Sharing Transformer

Area of Science:

• Computer Vision
• Deep Learning
• Image Processing

Background:

• Intrinsic image decomposition aims to separate an image into reflectance and shading components.
• Estimating consistent reflectance is challenging due to illumination variations affecting material appearance.
• Existing methods struggle with global and local reflectance consistency.

Purpose of the Study:

• To develop a novel deep neural network, CRefNet, for accurate and consistent reflectance estimation.
• To improve both global and local reflectance consistency in intrinsic image decomposition.
• To advance the state-of-the-art in intrinsic image decomposition.

Main Methods:

• CRefNet employs a hybrid transformer-convolutional architecture.
• A novel transformer module converts image features to reflectance features, capturing long-range interactions.
• An auxiliary reflectance reconstruction task and a rectified gradient filter are introduced.

Main Results:

• CRefNet achieves enhanced global reflectance consistency through its transformer module.
• The auxiliary task and gradient filter significantly improve reflectance map quality and local consistency.
• CRefNet outperforms state-of-the-art methods by 10% WHDR.

Conclusions:

• CRefNet effectively addresses the challenge of consistent reflectance estimation.
• The proposed methods enhance both global and local reflectance consistency.
• CRefNet represents a significant advancement in intrinsic image decomposition.