Gesture-Controlled Robotic Arm for Agricultural Harvesting Using a Data Glove with Bending Sensor and OptiTrack...

Area of Science:

• Robotics
• Agricultural Technology
• Human-Computer Interaction

Background:

• Fruit harvesting is labor-intensive, posing challenges for agricultural sustainability.
• Existing automation solutions often lack the dexterity or user-friendliness required for complex tasks.

Purpose of the Study:

• To develop a gesture-controlled robotic arm system for efficient and user-friendly agricultural harvesting.
• To leverage machine learning for accurate gesture recognition and robotic arm control.

Main Methods:

• Utilized a data glove with bending sensors and OptiTrack systems for gesture and spatial data capture.
• Employed a Convolutional Neural Network (CNN) combined with a Bidirectional Long Short-Term Memory (BiLSTM) model for gesture recognition.
• Integrated gesture recognition with robotic arm control for automated harvesting tasks.

Main Results:

• Achieved high precision in replicating hand movements, with a Euclidean Distance of 0.0131 m and RMSE of 0.0095 m.
• Demonstrated robust gesture recognition accuracy, reaching an overall accuracy of 96.43%.
• Validated the system's effectiveness in a simulated agricultural harvesting context.

Conclusions:

• The developed gesture-controlled robotic arm system presents a viable hybrid solution for agricultural harvesting.
• This technology enhances labor efficiency and sustainability in agriculture through precise and intuitive control.
• The system offers a promising approach for semi-automated agricultural practices, bridging the gap between manual and fully automated methods.