评估影响分数的因素 开放式响应情境判断测试:一种混合方法方法
Muhammad Zafar Iqbal1, Rodica Ivan1, Colleen Robb1
1Research Department, Acuity Insights, Toronto, ON, Canada.
Frontiers in medicine
|January 21, 2025
概括
情境判断测试 (SJT) 通过评估响应的内容和构造,有效地衡量专业性. 这项研究探讨了影响SJT得分的因素,证实了它们在评估专业能力方面的有效性.
科学领域:
- 心理测量评估是一种心理评估.
- 专业性评价 评价 专业性评价
- 教育心理学教育心理学
背景情况:
- 情境判断测试 (SJT) 在招生中被广泛用于评估专业性.
- 开放响应的SJT显示出希望,但对评估者评估的构造有效性未得到充分探索.
- 评估者的主观性可能会影响在SJT中对各种响应的评估.
研究的目的:
- 调查影响开放响应SJT评估员评估的构造相关和无关因素.
- 解决了解开放响应SJTs的构造有效性的差距.
- 探索评估人员如何看待和评估测试者响应中的专业性.
主要方法:
- 混合方法方法结合了对160个响应的定量内容分析 (Casper SJT) 和23名评估员的定性思维活动.
- 研究I:对应和后勤回归分析以确定响应得分的预测因素.
- 研究II:对转录的"大声思考"协议进行主题分析,以观察评估者评估过程.
主要成果:
- 构造相关的因素对得分产生了重大影响,包括能力的展示,与困境背景的接触,理由深度,对观点的考虑和解决方案的创造力.
- 构造无关的因素,如响应流和措辞,在影响分数方面显示出混合的结果.
- 评估人员的评价受到测试者答案的内容和构建的影响.
结论:
- 这项研究增强了开放响应SJT的构造有效性证据.
- 调查结果证实,SJT对于测量专业能力是有效的,考虑到响应的内容和结构.
- 该研究提供了对影响SJT评估评估者判断的因素的见解.
相关概念视频
Surveys
14.7K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
14.7K
Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
Attribution Theory
12.9K
Behavior is a product of both the situation (e.g., cultural influences, social roles, and the presence of bystanders) and of the person (e.g., personality characteristics). Subfields of psychology tend to focus on one influence or behavior over others. Situationism is the view that our behavior and actions are determined by our immediate environment and surroundings. In contrast, dispositionism holds that our behavior is determined by internal factors (Heider, 1958).
12.9K
Response Surface Methodology
88
Response Surface Methodology (RSM) is a collection of statistical and mathematical techniques used to develop, improve, and optimize processes. It is particularly valuable when many input variables or factors potentially influence a response variable.
The process of RSM involves several key steps:
The process of RSM involves several key steps:
88
Factors Affecting Perception
1.5K
Perception is influenced by perceptual set, context, motivation, and emotion. Perceptual set, or perceptual expectancy, refers to the tendency to perceive things in a particular way, influenced by previous experiences and expectations. This phenomenon affects the interpretation of stimuli, creating a set of mental tendencies and assumptions that impact sensory perceptions of sound, taste, touch, and sight.
An illustrative example of a perceptual set is the scenario where an airline pilot told...
An illustrative example of a perceptual set is the scenario where an airline pilot told...
1.5K
Self-Report Tests of Personality
305
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
305


