Human Shadows in Machine Minds: Quantitative Study Interpreting AI Responses to the Rorschach Test.

Area of Science:

• Artificial Intelligence
• Psychological Assessment
• Computational Linguistics

Background:

• Multimodal large language models (LLMs) exhibit humanlike capabilities in image description and dialogue.
• Adapting psychological assessment methods for LLMs is crucial for evaluating their behavior, especially under ambiguity.
• Projective tests, like the Rorschach inkblot test, have been underexplored in LLM research.

Purpose of the Study:

• To assess the feasibility of administering a full Rorschach protocol to multimodal LLMs.
• To descriptively compare response features of different LLMs using established Rorschach coding categories.

Main Methods:

• Administered all 10 standard Rorschach cards to three multimodal LLMs (GPT-4o, Grok 3, Gemini 2.0 Flash Thinking).
• Utilized standard and fallback prompts, followed by an inquiry phase for codable responses.
• Coded responses using the Exner Comprehensive System, analyzing response count, location, determinants, and human-related content.

Main Results:

• GPT-4o completed the protocol with a standard prompt; Grok 3 and Gemini required a fallback prompt.
• Response counts varied: GPT-4o (15), Grok 3 (10), Gemini (20).
• GPT-4o and Grok 3 predominantly produced whole-blot responses with more human movement determinants, while Gemini focused on common details.

Conclusions:

• Multimodal LLMs can generate Rorschach-like narratives mappable to standard coding categories.
• LLM outputs are sensitive to prompting and platform constraints, not indicative of an 'inner world'.
• LLM-assisted coding showed limitations; LLM response phenotypes deviate from human normative patterns.