Can foundation models reliably identify spatial hazards? A case study on curb segmentation.

Area of Science:

• Computer Vision
• Assistive Technology
• Robotics

Background:

• Curbs define safe pedestrian areas but pose stumbling hazards, especially for persons with blindness and low vision (PBLV).
• Accurate curb detection is crucial for developing visual-based assistive technologies for urban navigation.
• Current foundation models show limitations in precisely identifying curbs.

Purpose of the Study:

• To evaluate the effectiveness of foundation models for curb segmentation.
• To introduce a large-scale dataset for benchmarking curb segmentation models.
• To identify challenges and propose solutions for improving curb detection in assistive technology.

Main Methods:

• Development of the largest curb segmentation dataset to date.
• Benchmarking state-of-the-art foundation models on curb segmentation tasks.
• Analysis of model performance metrics including precision, recall, and inference time.
• Proposal of filtered bounding box selections for enhanced accuracy.

Main Results:

• State-of-the-art foundation models exhibit low precision and recall in curb segmentation.
• Models frequently confuse curbs with similar objects or non-curb surfaces.
• The best-performing model had an average inference time of 3.70 seconds, hindering real-time application.
• Filtered bounding box techniques show potential for improving accuracy.

Conclusions:

• Foundation models, while flexible, require significant refinement for practical assistive navigation.
• Specialized datasets and tailored training are essential for addressing PBLV navigation challenges.
• Current foundation models have implicit weaknesses in accurately segmenting curbs for assistive technologies.