Integrating Textual Queries with AI-Based Object Detection: A Compositional Prompt-Guided Approach

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Object detection and recognition are crucial for decision-making applications.
• Deep learning and language models offer new possibilities but face challenges in contextual query analysis and human interaction.

Purpose of the Study:

• To present a novel neuro-symbolic object detection framework.
• To enhance object detection and scene understanding through integrated deep learning and symbolic reasoning.

Main Methods:

• A neuro-symbolic framework aligning object proposals with textual prompts using deep learning.
• Integration of a deep learning module for object proposal alignment and a symbolic module for logical reasoning.
• Utilized a synthetic 3D image dataset for evaluation.

Main Results:

• The framework effectively generalizes to complex queries, combining simple attribute-based descriptions.
• Demonstrated enhanced object detection and scene understanding capabilities.
• Showcased the ability to handle compound prompts without explicit training.

Conclusions:

• The proposed neuro-symbolic framework significantly enhances object detection and scene understanding.
• This approach enables complex, query-driven interactions for emerging smart applications.
• Highlights the potential of integrating deep learning with symbolic reasoning for advanced AI systems.