Related Experiment Video
Updated: May 6, 2026
16:14
Trajectory Data Analyses for Pedestrian Space-time Activity Study
Published on: February 25, 2013
14.3K
DVS-PedX: Synthetic-and-Real Event-Based Pedestrian Dataset
Mustafa Sakhai1, Kaung Sithu2, Min Khant Soe Oke2
1Faculty of Computer Science, Electronics and Telecommunications, AGH University of Science and Technology, 30-059, Krakow, Poland. msakhai@agh.edu.pl.
Scientific Data
|March 5, 2026
Summary
A new dataset, DVS-PedX, uses event cameras for pedestrian detection and intention analysis in various conditions. Baseline Spiking Neural Network models show promising results, highlighting a need for domain adaptation in event-based perception research.
Area of Science:
- Neuromorphic Engineering
- Computer Vision
- Robotics
Background:
- Event cameras, such as Dynamic Vision Sensors (DVS), capture brightness changes asynchronously, offering advantages in low latency, high dynamic range, and motion robustness compared to traditional frame-based cameras.
- Pedestrian detection and intention prediction are critical for autonomous systems, especially in challenging environmental conditions like adverse weather and varying lighting.
Purpose of the Study:
- Introduce DVS-PedX, a novel neuromorphic dataset for pedestrian detection and crossing-intention analysis.
- Facilitate research in event-based perception for enhanced pedestrian safety and intention prediction systems.
- Provide a comprehensive resource for studying the sim-to-real gap in event-based vision.
Main Methods:
- DVS-PedX comprises two sources: synthetic event streams from CARLA simulator (198 sequences) and real-world JAAD dash-cam videos converted to event streams using v2e (346 clips).
- Data includes paired RGB frames, DVS event frames (33 ms accumulations), and binary labels for crossing events.
- Baseline experiments utilized Spiking Neural Networks (SNNs) implemented with SpikingJelly for performance evaluation.
Main Results:
- Spiking Neural Networks achieved an F1-score of 86.37% on the synthetic validation set.
- The study identified a significant sim-to-real gap, indicating challenges in transferring models trained on synthetic data to real-world scenarios.
- The dataset provides raw event files (AEDAT 2.0/4.0), DVS video files, and metadata for flexible data processing and model development.
Conclusions:
- DVS-PedX is a valuable resource for advancing event-based pedestrian safety and intention prediction research.
- The observed sim-to-real gap underscores the importance of domain adaptation and multimodal fusion techniques for robust event-based perception.
- The dataset aims to accelerate the development of neuromorphic perception systems for autonomous applications.
