The acceptability and validity of AI-generated psycholinguistic stimuli

Sentence stimuli pervade psycholinguistics research. Yet, limited attention has been paid to the automatic construction of sentence stimuli. Given their linguistic capabilities, this study investigated the efficacy of ChatGPT in generating sentence stimuli and AI tools in producing auditory sentence stimuli. In three psycholinguistic experiments, this study examined the acceptability and validity of AI-formulated auditory sentences and written sentences in one of the two languages: English and Arabic. In Experiment 1 and 3, participants gave English AI-generated stimuli similar to or higher acceptability ratings than human-composed stimuli. In Experiment 2, Arabic AI-generated stimuli received lower acceptability ratings than their human-composed counterparts. The validity of AI-developed stimuli relied on the study design, with only Experiment 2 demonstrating the target psycholinguistic effect. These results highlight the promising role of AI as a stimuli developer, which could facilitate psycholinguistic research and increase its diversity. Implications for psycholinguistic research were discussed.