Modality Exploration, Retrieval and Adaptation for Trajectory Prediction

Predicting future trajectories of dynamic agents is inherently riddled with uncertainty. Given a certain historical observation, there are multiple plausible future movements people can perform. Notably, these possible movements are usually centralized around a few representative motion patterns, e.g. acceleration, deceleration, turning, etc. In this paper, we propose a novel prediction scheme which explores human behavior modality representations from real-world trajectory data to discover such motion patterns and further uses them to aid in trajectory prediction. To explore potential behavior modalities, we introduce a deep feature clustering process on trajectory features and each cluster can represent a type of modality. Intuitively, each modality is naturally a class, and a classification network can be adopted to retrieve highly probable modalities about to happen in the future according to historical observations. On account of a wide variety of cues existing in the observation (e.g. agents' motion states, semantics of the scene, etc.), we further design a gated aggregation module to fuse different types of cues into a unified feature. Finally, an adaptation process is proposed to adapt a certain modality to specific historical observations and generate fine-grained prediction results. Extensive experiments on four widely-used benchmarks show the superiority of our proposed approach.