StreetAware: A High-Resolution Synchronized Multimodal Urban Scene Dataset

Area of Science:

• Urban computing
• Computer vision
• Robotics

Background:

• High-quality data is crucial for digital urban analysis, but access remains a significant barrier.
• Urban environments, especially street intersections, generate valuable sensor data for understanding dynamics between vehicles, pedestrians, and infrastructure.

Purpose of the Study:

• To introduce a novel, high-resolution, synchronized multimodal dataset of urban intersections.
• To provide researchers with a valuable resource for advancing machine learning applications in urban sensing and design.

Main Methods:

• Collected nearly 8 hours of synchronized audio, video, and LiDAR data from three Brooklyn, New York urban intersections.
• Utilized custom Reconfigurable Environmental Intelligence Platform (REIP) sensors for accurate synchronization of multiple data streams.
• Ensured data is multimodal, multi-angular, high-resolution, and synchronized.

Main Results:

• Demonstrated four key data utilization methods: occluded object detection, audio-visual event association, object tracking, and pedestrian speed measurement.
• The dataset facilitates novel analyses in pedestrian-vehicle interaction modeling and pedestrian attribute recognition.
• The data supports research in accessibility-aware urban design and Vision Zero initiatives.

Conclusions:

• The presented dataset addresses the need for high-quality, synchronized multimodal data in urban environments.
• This resource empowers researchers to develop advanced machine learning models for urban sensing and smart city applications.
• The data can significantly contribute to improving urban safety, accessibility, and design.