Design of jitter compensation algorithm for robot vision based on optical flow and Kalman filter

Area of Science:

• Robotics
• Computer Vision
• Image Processing

Background:

• Image jitters in autonomous robot videos reduce visual operation precision.
• Compensating for these jitters is crucial for enhancing robot performance.

Purpose of the Study:

• To establish affine transformation kinematics for obtaining six image motion parameters.
• To develop a feature point detection method and solve motion parameters using least squares and optical flow.
• To analyze the impact of matching errors on parameter solving and smooth motion parameters using a Kalman filter.

Main Methods:

• Established affine transformation kinematics to derive six image motion parameters.
• Designed a feature point pair detection method based on Eigen-value analysis.
• Solved motion parameters using the least square method with point pairs obtained via optical flow.
• Utilized the condition number of the coefficient matrix to analyze matching errors.
• Employed a Kalman filter for smoothing estimated image motion parameters.

Main Results:

• More matching point pairs lead to more precise motion parameter solutions.
• The integrated jitter compensation software, featuring sub-window feature detection, was developed.
• Practical experiments on mobile robots demonstrated effective jitter compensation.
• The compensation process time was found to be less than the frame sample time.

Conclusions:

• The developed method effectively compensates for robot vision jitters.
• Kalman filtering is a valid technique for smoothing image motion parameters in robotic applications.
• The system enhances robot operation precision by mitigating image jitter effects.