Related Experiment Video
Updated: Aug 5, 2026
06:04
Functional Near-Infrared Spectroscopy Hyperscanning Study in Psychological Counseling
Published on: January 17, 2025
Multi-Agent LLMs for Occupational Profiling: Psychometric Validation on 1636 Chinese Occupations
Yuting Han1, Xiaoyang Luo2, Feng Ji3
1Cognitive Science and Allied Health School, Institute of Life and Health Sciences, Key Laboratory of Language and Cognitive Science (Ministry of Education), Beijing Language and Culture University, Beijing 100083, China.
Behavioral Sciences (Basel, Switzerland)
|July 28, 2026
Summary
A new multi-agent large language model (LLM) framework efficiently generates occupation-level psychological profiles, creating the first comprehensive RIASEC and Big Five personality database for Chinese occupations.
Area of Science:
- Psychometrics and Computational Social Science.
- Leveraging artificial intelligence for large-scale psychological assessment.
Background:
- Occupation-level psychological profiles (RIASEC, Big Five) are crucial for career counseling and workforce research.
- Previous methods for building these profiles were costly and limited in scope.
- Existing large language models (LLMs) face challenges in reliability, calibration, and validation for psychometric rating.
Purpose of the Study:
- To develop a scalable and reliable method for generating occupation-level psychological profiles using LLMs.
- To create the first comprehensive database of RIASEC and Big Five scores for all occupations in the 2022 Chinese Occupational Classification.
- To address inter-rater reliability, calibration, and validation issues in LLM-based psychometric assessments.
Main Methods:
- Proposed a multi-agent LLM framework with three LLMs as expert raters, in-context anchors for scale alignment, and an arbitrator for disagreement resolution.
- Applied the framework to all 1636 occupations in the 2022 Chinese Occupational Classification.
- Generated six RIASEC and five Big Five scores per occupation, assessing reliability and validity.
Main Results:
- Achieved excellent reliability for RIASEC dimensions (ICC [2,1] = 0.87–0.98) and high convergent correlations with O*NET (r = 0.84–0.96).
- Big Five scores demonstrated high absolute agreement; structural validity was weakly supported but differentiated occupational categories.
- The framework produced the first occupation-level RIASEC and Big Five database for the Chinese Classification, openly available.
Conclusions:
- The multi-agent LLM framework offers a scalable and reliable solution for generating occupation-level psychological profiles.
- The resulting database provides valuable resources for career counseling, person-job matching, and workforce research in China.
- LLM-based psychometric assessments can overcome previous limitations, though careful validation and calibration are essential.
