Ranking-Based Salient Object Detection and Depth Prediction for Shallow Depth-of-Field

Area of Science:

• Computer Vision
• Computational Photography
• Image Processing

Background:

• Achieving shallow depth-of-field (DoF) is a complex task in computer vision and computational photography.
• Existing methods often require specific camera settings or are limited to portrait images with stereo inputs.

Purpose of the Study:

• To develop a computational method for rendering shallow DoF from arbitrary images.
• To overcome the limitations of prior techniques that rely on portrait segmentation or stereo sensing.

Main Methods:

• A novel method combining salient object detection (SOD) and monocular depth prediction (MDP) modules.
• Introduction of a label-guided ranking loss for both SOD and MDP, incorporating heterogeneous and homogeneous ranking for SOD, and multilevel structural information for MDP.
• Development of a SOD and depth-aware blur rendering technique for shallow DoF image generation.

Main Results:

• The proposed method effectively renders shallow DoF effects on arbitrary images.
• Experiments demonstrate the superiority of the SOD and MDP modules in determining focal planes and controlling blur.
• The label-guided ranking loss significantly improves both salient object detection and depth prediction accuracy.

Conclusions:

• The developed computational approach successfully generates shallow DoF images from single, arbitrary inputs.
• The method offers a versatile alternative to traditional DoF techniques, applicable beyond portrait photography.
• The integration of SOD, MDP, and a novel ranking loss provides a robust framework for advanced image manipulation.