Quantitative Representation of Autonomous Driving Scenario Difficulty Based on Adversarial Policy Search

Autonomous vehicles with self-evolution capabilities are expected to improve their performance through learning algorithms, to automatically adapt to the external environment. However, due to the infinity, complexity, and variability of the actual traffic environment, it is necessary to develop quantitative representation indicators of scenario difficulty and generate targeted scenarios to ensure the evolution gradually, so as to quickly approach the performance limit of the algorithm. Therefore, this paper proposes a data-driven quantitative representation method of scenario difficulty. Specifically, the concept of environment agent is proposed, and a reinforcement learning method combined with mechanism knowledge is constructed for policy search to obtain an agent with an adversarial behavior. The model parameters of the environment agent at different stages in the training process are extracted to construct a policy group, and then agents with different adversarial intensities are obtained, which are used to realize data generation in different difficulty scenarios through the simulation environment. Finally, a data-driven scenario difficulty quantitative representation model is constructed, which is used to output the environment agent policy under different difficulties. Experimental results show the effectiveness of the proposed method. The result analysis shows that the proposed algorithm can generate reasonable and interpretable scenarios with high discrimination and can provide quantifiable difficulty representation without any expert logic rule design. Compared with the rule-based discrete scenario difficulty representation method, the proposed algorithm can achieve continuous difficulty representation. The video link is https://www.youtube.com/watch?v=GceGdqAm9Ys.