Image Generation for 2D-CNN Using Time-Series Signal Features from Foot Gesture Applied to Select Cobot Operating...

Area of Science:

• Robotics and Human-Computer Interaction
• Machine Learning for Industrial Automation

Background:

• Manufacturing tasks can be burdensome for workers, necessitating intuitive human-robot collaboration.
• Cobots can act as a 'third-arm' to assist in assembly, requiring new control modalities.

Purpose of the Study:

• To develop and evaluate a foot gesture recognition system for controlling industrial robot operating modes.
• To investigate the effectiveness of using an instrumented insole and CNNs for intuitive robot control.

Main Methods:

• An instrumented insole with an inertial measurement unit (IMU) and force sensors acquired foot gesture data.
• Statistical feature extraction and variance analysis (ANOVA) reduced features from 78 to 3.
• Time-series data were converted into 2D images for input into a 2D convolutional neural network (CNN).

Main Results:

• The CNN achieved effective recognition of foot gestures when data features were represented as 2D images.
• Gesture recognition rates were highly dependent on feature selection and their spatial representation.
• Specific feature representations using triangular and rectangular forms yielded higher recognition rates.

Conclusions:

• CNNs show promise for recognizing foot gestures for industrial robot control.
• The developed foot gesture control scheme offers an intuitive method for switching cobot operating modes.
• This technology can enhance human-robot collaboration and reduce worker burden in manufacturing.