Bayesian teaching enables probabilistic reasoning in large language models

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Machine Learning

Background:

• Large language models (LLMs) are increasingly deployed as agents interacting with users and the environment.
• Effective agent behavior necessitates constructing world representations and probabilistic beliefs.
• Personalized recommendations require LLMs to infer user preferences from interaction history.

Purpose of the Study:

• To evaluate the belief-updating capabilities of LLMs against the normative Bayesian inference framework.
• To investigate whether teaching LLMs to mimic Bayesian models enhances their reasoning and generalization skills.

Main Methods:

• LLMs' belief-updating performance was assessed against Bayesian standards.
• LLMs were trained to emulate predictions from a normative Bayesian model.
• The generalization of learned belief-updating skills to novel tasks was evaluated.

Main Results:

• LLMs demonstrated significant deficiencies in adhering to the Bayesian framework for belief updating.
• Training LLMs to mimic Bayesian predictions led to substantial improvements in belief updating.
• The enhanced belief-updating ability generalized effectively to unseen tasks.

Conclusions:

• LLMs exhibit limitations in optimal belief updating compared to Bayesian agents.
• LLMs can acquire and generalize reasoning skills, specifically belief updating, through imitation learning of Bayesian models.
• This research highlights the potential for LLMs to learn complex reasoning abilities from data.