Classification of the Sidewalk Condition Using Self-Supervised Transfer Learning for Wheelchair Safety Driving

Area of Science:

• Computer Vision
• Robotics
• Accessibility Engineering

Background:

• Increasing demand for wheelchairs highlights the need for safe infrastructure.
• Existing sidewalk condition assessment methods using RGB images or IMU sensors are unreliable in adverse outdoor conditions.
• Cracks and potholes in sidewalks pose significant threats to wheelchair users' safety.

Purpose of the Study:

• To develop a robust system for automatic sidewalk condition classification.
• To evaluate the performance of various convolutional neural networks (CNNs) using depth and infrared imaging.
• To compare training CNNs from scratch versus transfer learning approaches for this task.

Main Methods:

• Utilized depth and infrared camera modalities for image capture.
• Implemented and compared various convolutional neural network (CNN) architectures.
• Investigated transfer learning, including fine-tuning ImageNet-pre-trained models and using ResNet-152 pre-trained with self-supervised learning.
• Evaluated performance using 100% and 10% of the training data.

Main Results:

• The proposed system demonstrated effectiveness and feasibility in classifying sidewalk conditions.
• Transfer learning, particularly with self-supervised pre-trained ResNet-152, showed promise for improved image representation.
• The system offers a more stable and accurate alternative to existing methods.

Conclusions:

• The novel system using CNNs with depth and infrared data successfully classifies sidewalk conditions.
• Transfer learning enhances the performance of sidewalk hazard detection systems.
• This research paves the way for improved accessibility infrastructure and future advancements in the field.