Task-Decomposed Language Supervision for End-to-End Autonomous Driving

End-to-end (E2E) autonomous driving integrates perception, prediction, and planning, yet often lacks explicit supervision for high-level driving semantics. We propose Task-Decomposed Language Supervision (TaLS), a training-only framework that organizes DriveLM question-answer annotations into four sequential stages: Object/Scene, Motion, Risk, and Planning. During training, the answer-end hidden state of each stage is projected into designated driving-task queries through gate-free residual injection. The autoregressive language model and injections are removed at inference, while the BEV token modules remain active. On the nuScenes open-loop benchmark, the TaLS reference run reduces the TemAvg box-collision metric by 31.3% relative to a matched GenAD baseline with comparable trajectory L2 and perception performance. Across three seeds, TemAvg improvements occur in two runs, with closely matched mean collision rates; under NoAvg, TaLS achieves an equal or lower NoAvg collision rate in all three runs. Component ablations identify stage-wise injection and decomposition as contributors to the observed collision reduction, and a pruning analysis separates the roles of retained BEV token modules and training-time language supervision. TaLS also maintains a lower absolute collision metric under four tested camera degradations. These findings illustrate how structured language supervision can shape open-loop planning behavior without autoregressive language-model inference.