TOSD: A Hierarchical Object-Centric Descriptor Integrating Shape, Color, and Topology

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Existing pixel-based and global feature embedding methods have limitations in capturing complex scene semantics.
• A need exists for a descriptor framework that supports multi-level reasoning and integrates diverse object attributes.
• Previous work on the Semantic Modeling Framework established layered environment representations.

Purpose of the Study:

• To introduce the Triplet Object-Centric Semantic Descriptor (TOSD), a hierarchical framework for object-centric scene representation.
• To overcome limitations of current visual feature descriptors by integrating shape, color, and topological information.
• To enable multi-level reasoning from low-level pixel details to high-level semantic structures.

Main Methods:

• Developed a hierarchical object-centric descriptor framework (TOSD).
• Integrated shape, color, and topological information at object and scene levels.
• Created a representation with three abstraction levels: pixel, object, and semantic structure.

Main Results:

• TOSD demonstrated competitive performance across multiple computer vision tasks.
• The framework showed robustness against challenges like occlusion and viewpoint changes.
• Achieved compact and consistent embeddings integrating local cues and global context.

Conclusions:

• TOSD provides a unified and effective approach to hierarchical visual scene representation.
• The method is versatile and applicable to a wide range of vision and robotics tasks.
• This work advances semantic scene understanding through object-centric, multi-level feature integration.