Transformer-based Light Field Salient Object Detection and Its Application to Autofocus.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Existing light field salient object detection (LFSOD) models struggle with long-range dependency modeling in focal stacks.
• Convolutional Neural Networks and local attention limit the ability to capture complex relationships within light field data.

Purpose of the Study:

• To propose a novel, quasi-pure Transformer-based framework (TLFNet) for light field salient object detection.
• To address limitations in modeling intra-slice and cross-slice dependencies in light field data.
• To introduce a new application for autonomous bokeh effects in photography.

Main Methods:

• Developed TLFNet, a framework utilizing Transformer-based fusion modules (PGFormer) with perpendicular self-attention (PSA) for long-range dependency capture.
• Integrated a guided feature fusion (GFF) module for multi-modal feature integration.
• Incorporated an edge enhancement module with focal loss to improve salient object edge precision.

Main Results:

• TLFNet significantly outperforms 14 existing light field models, establishing new state-of-the-art performance.
• The Transformer-based architecture (99.01% parameters) effectively models dependencies, while the edge enhancement module (0.99% parameters) boosts accuracy.
• Demonstrated a new application, light field salient object autofocus (LFSOA), by combining TLFNet with deep autofocus techniques.

Conclusions:

• TLFNet offers a highly effective, Transformer-centric approach to light field salient object detection.
• The proposed method achieves superior performance and introduces innovative applications like autonomous bokeh.
• The framework demonstrates the potential of pure Transformer architectures in complex visual tasks.