Divergent creativity in humans and large language models

Area of Science:

• Computational Creativity
• Cognitive Science
• Artificial Intelligence

Background:

• Large Language Models (LLMs) are increasingly claimed to possess human-like creativity.
• A systematic evaluation of LLMs' semantic diversity compared to human divergent thinking is lacking.
• Divergent thinking, involving associative thinking and remote concept combination, is a key aspect of creative cognition.

Purpose of the Study:

• To systematically evaluate and compare the semantic diversity of state-of-the-art LLMs against a large human dataset.
• To benchmark LLM performance on creative tasks using objective measures.
• To explore methods for enhancing LLM semantic divergence.

Main Methods:

• Leveraged computational creativity techniques to analyze semantic divergence in LLMs and 100,000 human participants.
• Utilized the Divergent Association Task (DAT) and creative writing tasks (haiku, story synopses, flash fiction) for benchmarking.
• Employed identical, objective scoring and systematically varied linguistic strategy prompts and temperature settings.

Main Results:

• LLMs surpassed average human performance on the DAT and approached human creative writing abilities.
• LLMs' performance remained below the mean creativity scores of the more creative human participants.
• Top-performing LLMs were significantly surpassed by the aggregated top half of human participants, indicating a performance ceiling.
• Varying linguistic prompts and temperature settings reliably improved semantic divergence in several LLMs.

Conclusions:

• LLMs demonstrate significant progress in creative linguistic tasks but do not yet match the highest levels of human creativity.
• The study provides a framework for objectively comparing human and AI creative output, addressing concerns about AI replacing human creative labor.
• Techniques like prompt design and hyper-parameter tuning can enhance LLM semantic diversity, offering avenues for future AI development.