Do Language Models Know Who Did What to Whom?

Area of Science:

• Computational linguistics
• Cognitive science
• Artificial intelligence

Background:

• Language models (LMs) face criticism for lacking true language understanding.
• Existing critiques often focus on cognitive abilities separate from language processing.

Purpose of the Study:

• To investigate whether the word prediction objective in LMs leads to representations capturing thematic roles.
• To assess the extent to which LMs understand sentence structure in terms of agent/patient relationships.

Main Methods:

• Characterized sentence representations in four LMs proposed as models of human language processing.
• Conducted two experiments to analyze representational similarity and thematic role availability in hidden activations.
• Examined the role of specific attention heads in capturing thematic information.

Main Results:

• Overall sentence representational similarity did not correlate with shared or opposing agent/patient assignments.
• Limited evidence found for thematic role information within general hidden activation subspaces.
• Certain attention heads demonstrated robust capture of thematic roles, independent of syntactic structure.

Conclusions:

• LMs possess the capability to extract thematic roles from sentences.
• Thematic role information has a weak influence on the overall sentence representations within LMs.
• Further research is needed to understand how this extracted information is utilized or integrated.