Successive-cyclic movement in humans and neural language models: testing wh-filler-gap dependencies

This study investigates whether auto-regressive language models (GPT-2, GPT-Neo, OPT) replicate human-like sensitivity to covert intermediate phrasal structures (CP vs. NP) during the processing of wh-filler-gap dependencies. We extend this inquiry to backward sluicing, an elliptical construction that provides a robust test for the representation of abstract syntactic structure. Across two experiments measuring processing difficulty via surprisal, we found a significant divergence from established human processing patterns. We found that the models failed to reproduce the human processing facilitation for both canonical and elided dependencies. One model, in fact, showed an inverse effect, a pattern suggesting a reliance on surface-level cues rather than abstract hierarchical representations. We take these findings as evidence that the tested GPT-style models are insufficient for deriving knowledge of covert syntactic structures. This failure lends empirical support to the Poverty of the Stimulus (PoS) argument, and also highlights a significant gap in the cognitive plausibility of contemporary NLMs as models of human syntactic competence.