Pictionary-Style Word Guessing on Hand-Drawn Object Sketches: Dataset, Analysis and Deep Network Models

Area of Science:

• Artificial Intelligence
• Computer Vision
• Human-Computer Interaction

Background:

• Playing games in a human-like manner is a key benchmark for Artificial Intelligence (AI) progress.
• Pictionary, a popular social word-guessing game, presents unique challenges for AI due to its reliance on visual interpretation and sequential information.

Purpose of the Study:

• To introduce the first computational model specifically designed for the game of Pictionary.
• To develop an AI system capable of generating human-like guess-words from evolving sketch data.
• To analyze a new dataset, Sketch-QA, created for evaluating Pictionary-style AI.

Main Methods:

• Introduction of Sketch-QA, a novel guessing task using incrementally accumulated sketch stroke sequences as visual input.
• Development of a deep neural network model to generate guess-words in response to temporally evolving sketches.
• Evaluation of the model on the large-scale Sketch-QA dataset and comparison with baseline methods.
• Conducting a Visual Turing Test to compare AI-generated guesses with human guesses.

Main Results:

• The proposed deep neural model successfully generates guess-words from sketches, mimicking human-like guessing patterns.
• The model exhibits human-like errors, enhancing its ability to replicate human performance in Pictionary.
• Analysis of the Sketch-QA dataset revealed interesting findings regarding human guessing strategies.
• The Visual Turing Test indicated promising results for the AI's ability to generate plausible guesses.

Conclusions:

• The developed computational model shows significant promise for Pictionary and similar visual guessing games.
• The approach advances AI's capability in understanding and generating responses based on dynamic visual input.
• The Sketch-QA task and dataset provide a valuable resource for future research in AI game playing and visual reasoning.