Artificial Intelligence Models Do Not Ground Negation, Humans Do. GuessWhat?! Dialogues as a Case Study.

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Negation is crucial in human communication but often overlooked in neural network-based conversational agents.
• Cognitive research indicates that visual context can facilitate negation processing.

Purpose of the Study:

• To evaluate how well conversational agents, specifically those using pre-trained language models, utilize negatively answered polar questions in a visually grounded guessing game.
• To compare model performance with human performance in understanding and processing negation within a referential context.

Main Methods:

• Utilized the GuessWhat?! visually grounded guessing game as a testbed.
• Evaluated pre-trained language model-based guessers on their ability to process negative questions.
• Conducted a crowdsourcing experiment with human subjects playing the same game to compare with model results.

Main Results:

• Humans effectively leverage negatively answered questions for task completion, whereas models demonstrate difficulty in grounding negation.
• Some models showed minimal utilization of negative linguistic cues.
• Visual features proved beneficial for models in encoding negative information when language signals were ambiguous.

Conclusions:

• Current conversational agents struggle to effectively process and utilize negation compared to humans.
• Visual context can partially mitigate models' difficulties with negation, especially in ambiguous linguistic situations.
• Comparing model errors to human-like errors provides insights into model plausibility and areas for improvement.