Hybrid FPGA-CPU-Based Architecture for Object Recognition in Visual Servoing of Arm Prosthesis

Area of Science:

• Robotics
• Computer Vision
• Biomedical Engineering

Background:

• Visual servoing is crucial for prosthetic arm control.
• Existing systems face challenges with real-time processing and low power consumption for wearable devices.

Purpose of the Study:

• To implement a hybrid hardware-software system for gaze-driven object recognition in prosthetic arms.
• To achieve low power consumption (<5.6 W) for a wearable device.

Main Methods:

• Developed a lightweight architecture for gaze-driven object recognition.
• Utilized an eye tracker and camera for visual input.
• Implemented computationally intensive parts (SIFT detector, CNN feature extractor) on FPGA.
• Introduced a novel reduction layer in the object-recognition CNN to decrease computational load.

Main Results:

• Achieved real-time object recognition for prosthetic arm control.
• The system demonstrates low power consumption, meeting the <5.6 W requirement.
• Hybrid hardware-software approach significantly reduces computational burden.

Conclusions:

• The proposed system effectively enables visual servoing of prosthetic arms.
• The implementation is suitable for wearable, low-power applications.
• Optimized vision analysis is key for real-time prosthetic control.