Quantifying uncert-AI-nty: Testing the accuracy of LLMs' confidence judgments

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Human-Computer Interaction

Background:

• Large Language Models (LLMs) like ChatGPT and Gemini are transforming information access.
• Metacognitive confidence judgments are crucial for human uncertainty quantification.
• The accuracy of LLM confidence judgments remains largely unexplored.

Purpose of the Study:

• To investigate the capability of LLMs to quantify uncertainty through confidence judgments.
• To compare the metacognitive accuracy of LLMs and humans across various tasks.
• To identify similarities and differences in confidence judgment strategies between LLMs and humans.

Main Methods:

• Four LLMs (ChatGPT, Bard/Gemini, Sonnet, Haiku) and human participants evaluated their confidence in predictions and answers.
• Studies covered aleatory uncertainty (NFL, Oscar predictions) and epistemic uncertainty (Pictionary, Trivia, university life questions).
• Absolute and relative accuracy of confidence judgments were analyzed.

Main Results:

• LLMs demonstrated comparable, and sometimes superior, absolute and relative metacognitive accuracy to humans.
• Both LLMs and humans exhibited overconfidence in their judgments.
• LLMs, especially ChatGPT and Gemini, often failed to adjust confidence based on prior performance, unlike humans.

Conclusions:

• LLMs possess significant capabilities in metacognitive confidence judgments, approaching human levels of accuracy.
• Overconfidence is a shared trait between LLMs and humans.
• A key limitation for LLMs is their reduced ability to dynamically adjust confidence based on experience, unlike humans.