Simulating human well-being with large language models: Systematic validation and misestimation across 64,000...

Area of Science:

• Computational Social Science
• Artificial Intelligence Ethics

Background:

• Subjective well-being is crucial for economic, medical, and policy decisions.
• Accurate measurement of well-being is essential for informed decision-making.

Purpose of the Study:

• To evaluate the validity of large language models (LLMs) in predicting global subjective well-being.
• To benchmark LLM performance against traditional statistical models and human self-reports.

Main Methods:

• Utilized natural-language profiles of 64,000 individuals across 64 countries.
• Compared predictions from four leading LLMs with self-reported well-being and regression models.
• Conducted a preregistered experiment to assess LLM reliance on linguistic associations versus conceptual understanding.

Main Results:

• LLMs produced plausible patterns but systematically underperformed compared to regression models.
• LLM errors were largest in underrepresented countries, reflecting digital and economic inequalities.
• LLMs demonstrated reliance on surface-level linguistic associations, leading to predictable distortions in unfamiliar contexts.

Conclusions:

• LLMs can simulate broad correlates of life satisfaction but lack experiential and cultural depth.
• LLMs are not suitable substitutes for human self-reports of well-being.
• Using LLMs for well-being assessment risks reinforcing inequality and undermining human agency.