人类专业知识和大型语言模型嵌入内容有效性评估的人格测试
Nicola Milano1, Michela Ponticorvo1, Davide Marocco1
1Department of Humanistic Studies, Natural and Artificial Cognition Laboratory "Orazio Miglino," University of Naples Federico II, Naples, Italy.
Educational and psychological measurement
|August 19, 2025
概括
大型语言模型 (LLM) 在评估诸如五大问卷 (BFQ) 和五大清单 (BFI) 等心理测量仪器方面表现有前途. 结合人类专业知识和人工智能的混合系统提供可扩展,客观的测试开发.
科学领域:
- 心理测量和心理评估
- 在行为科学中的人工智能
- 自然语言处理用于测量.
背景情况:
- 内容有效性至关重要,以确保心理测量准确地反映了预期的构造.
- 传统方法依赖于人类的专家判断,这可能是耗时和主观的.
- 心理构造的日益复杂性需要创新的验证方法.
研究的目的:
- 研究大语言模型 (LLM) 在评估心理测量仪器的内容有效性方面的应用.
- 为了比较LLMs与人类专家评估在语义项目构建对齐中的表现.
- 探索LLM培训策略对内容有效性评估准确性的影响.
主要方法:
- 人类专家 (心理学研究生) 使用内容有效率来评估五大问卷 (BFQ) 和五大库存 (BFI) 的项目.
- 最先进的LLM,包括多语言和微调模型,分析了项目嵌入来预测构造映射.
- 人类和人工智能驱动的方法之间的语义项目构建对齐准确性的比较.
主要成果:
- 人类验证器在行为丰富的BFQ项目中表现出更高的准确性.
- 在语言上简洁的BFI项目中,LLM表现优异.
- 法学士的表现受到培训策略的显著影响,词汇集中型模型的表现优于一般模型.
结论:
- 在心理测量仪器开发中,LLM为内容有效性评估提供了可扩展和客观的方法.
- 结合人类专业知识与人工智能精度的混合系统为强大的测试验证提供了一个有希望的方向.
- 法律学历有潜力改变心理评估方法,提高效率和客观性.
更多相关视频
相关概念视频
Introduction to Personality Psychology
17.7K
Personality encompasses a set of enduring traits and behavioral patterns that define how individuals think, feel, and interact, ultimately shaping their unique identities. The concept of personality has deep historical roots, deriving from the Latin term "persona," which means "mask." This term initially referred to the roles played by actors in ancient theater, signifying the different facets individuals display in various contexts.
Early Theories of Personality
The study of...
Early Theories of Personality
The study of...
17.7K
Stereotype Content Model
14.9K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.9K
Self-Report Tests of Personality
454
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
454
Five-Factor Theory of Personality
1.2K
The five-factor model, often called the Big Five personality traits, is widely accepted in psychology as a comprehensive framework for understanding personality. These five traits — Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism — are often remembered using the acronym OCEAN.
Openness reflects creativity, curiosity, and openness to new experiences. Individuals scoring high in openness are imaginative, have a wide range of interests, and are independent...
Openness reflects creativity, curiosity, and openness to new experiences. Individuals scoring high in openness are imaginative, have a wide range of interests, and are independent...
1.2K
Personality Theory by Eysenck and Eysenck
657
Hans and Sybil Eysenck developed a widely recognized theory of personality, which emphasizes the role of temperament and genetically based differences in shaping individual traits. Their theory posits that biological factors primarily determine personality and can be understood through two main dimensions: extroversion/introversion and neuroticism/stability.
In the extroversion/introversion dimension, highly extroverted people are sociable, outgoing, and easily connect with others. In contrast,...
In the extroversion/introversion dimension, highly extroverted people are sociable, outgoing, and easily connect with others. In contrast,...
657
Higher Mental Functions of the Brain: Language
1.0K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
1.0K


