Image Representations with Spatial Object-to-Object Relations for RGB-D Scene Recognition.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Scene recognition faces challenges due to high intra-class diversity and inter-class similarity.
• Existing methods often rely on global image features or object-level representations, limiting discriminative power.
• Object-to-object relations offer a more nuanced approach to understanding scene context.

Purpose of the Study:

• To develop more discriminative image representations for scene recognition by focusing on object-to-object relationships.
• To introduce two novel representations: co-occurring object-to-object relation (COOR) and sequential object-to-object relation (SOOR).
• To adapt these methods for RGB-D data to enhance spatial information capture.

Main Methods:

• Object detection techniques are used to extract triplets of .
• COOR is represented as a three-order tensor capturing the co-occurring frequency of object relations.
• SOOR is represented as sequential local captions, encoded using a recurrent neural network (RNN).
• A RGB-D proposal fusion method is employed for RGB-D object detection.

Main Results:

• The proposed COOR and SOOR methods significantly improve scene recognition accuracy.
• State-of-the-art results are achieved on the SUN RGB-D and NYUD2 datasets for RGB-D scene recognition.
• The methods effectively capture spatial information crucial for scene understanding.

Conclusions:

• Object-to-object relations provide highly discriminative features for scene recognition.
• COOR and SOOR are effective representations for scene classification, outperforming previous approaches.
• The adapted methods demonstrate strong performance on RGB-D scene recognition tasks.