Semantic scene upgrades for trajectory prediction

Understanding pedestrian motion is critical for many real-world applications, e.g., autonomous driving and social robot navigation. It is a challenging problem since autonomous agents require complete understanding of its surroundings including complex spatial, social and scene dependencies. In trajectory prediction research, spatial and social interactions are widely studied while scene understanding has received less attention. In this paper, we study the effectiveness of different encoding mechanisms to understand the influence of the scene on pedestrian trajectories. We leverage a recurrent Variational Autoencoder to encode pedestrian motion history, its social interaction with other pedestrians and semantic scene information. We then evaluate the performance on various public datasets, such as ETH-UCY, Stanford Drone and Grand Central Station. Experimental results show that utilizing a fully segmented map, for explicit scene semantics, out performs other variants of scene representations (semantic and CNN embedding) for trajectory prediction tasks.