Can Language Models Trained on Written Monologue Learn to Predict Spoken Dialogue?

Area of Science:

• Computational Linguistics
• Cognitive Science
• Artificial Intelligence

Background:

• Transformer-based Large Language Models (LLMs) demonstrate significant capabilities in text generation.
• The ability of LLMs to model spoken language, particularly in interactive contexts, remains underexplored.
• Spoken and written language exhibit distinct characteristics in syntax, pragmatics, and conversational norms.

Purpose of the Study:

• To evaluate Large Language Models (LLMs) as predictive models of spoken dialogue.
• To investigate whether LLMs can learn that speaker identity influences utterance predictability in conversation.
• To compare LLM predictions with human behavioral data in natural spoken interactions.

Main Methods:

• Fine-tuning two variants of GPT-2 on English spoken dialogue transcripts.
• Calculating word surprisal values for two-turn conversational sequences.
• Comparing model-generated surprisal data with human behavioral responses regarding speaker identity.

Main Results:

• Fine-tuned LLMs showed that word predictability is influenced by speaker identity.
• However, the models did not replicate the way humans utilize speaker identity information in predicting spoken language.
• LLM performance indicated a divergence from human conversational behavior.

Conclusions:

• LLMs can learn normative linguistic structures from spoken dialogue data.
• Current LLMs do not fully capture the pragmatic and social factors, such as speaker identity, that shape natural human conversation.
• Further research is needed to enhance LLMs' ability to model the complexities of interactive spoken language.