Integration of Industrially-Oriented Human-Robot Speech Communication and Vision-Based Object Recognition

Area of Science:

• Robotics
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Integrating speech and vision for industrial robots is crucial for context-aware commands.
• Existing research often overlooks vision-based object identification for general terms in industrial settings.
• Current systems face challenges due to differing data structures between speech and image recognition.

Purpose of the Study:

• To develop a novel method for seamless integration of human-robot speech communication and vision-based object recognition.
• To address the limitations of current systems in handling context-based voice control for industrial robots.
• To facilitate the creation of customized voice control applications for industrial robots.

Main Methods:

• Extension of the Voice Command Description (VCD) format to incorporate syntax and semantics of task-oriented commands.
• Integration of VCD with Flexible Editable Contour Templates (FECT) for classifying contours from image recognition.
• Developing a method to bridge the abstraction gap between speech and image recognition systems.

Main Results:

• A novel method for integrating speech and vision in industrial human-robot communication.
• Improved context-based speech communication, leading to shorter and more accurately recognized commands.
• A foundational solution for developing customized vision-based voice control for industrial robots.

Conclusions:

• The proposed method successfully integrates speech and vision, enhancing industrial robot command recognition.
• This research pioneers a solution for context-aware voice control in industrial robotics.
• The developed approach simplifies the creation of specialized vision-based voice control systems for robots.