Related Experiment Video
Updated: Dec 15, 2025
07:34
Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues
Published on: June 3, 2013
17.8K
Exploring the Impersonal Judgments and Personal Preferences of Raters in Rater-Mediated Assessments With Unfolding
1University of Miami, Coral Gables, FL, USA.
Educational and Psychological Measurement
|July 14, 2020
Summary
Unfolding models can evaluate rater-mediated assessments by distinguishing impersonal judgments from personal preferences. This method helps assess rating quality and understand rater decision-making processes.
Area of Science:
- Educational Measurement
- Psychometrics
- Assessment Quality Evaluation
Background:
- Rater-mediated assessments rely on judgments, ideally impersonal, to ensure rating quality.
- Raters may introduce personal preferences, potentially introducing construct-irrelevant variance.
- Understanding judgmental processes is crucial for improving assessment validity.
Purpose of the Study:
- To explore unfolding models as a framework for evaluating rating quality in rater-mediated assessments.
- To differentiate between impersonal judgments and personal preferences in rater responses.
- To investigate the utility of unfolding models in understanding rater decision-making.
Main Methods:
- Utilized unfolding models to analyze rating data from a large-scale US writing assessment.
- Examined essay features (e.g., word count, syntactic simplicity) in relation to rater preferences.
- Applied psychometric techniques to evaluate the judgmental processes underlying ratings.
Main Results:
- Unfolding models provide a valuable approach for initial exploration of judgmental processes in rater-mediated assessments.
- Significant relationships were found between specific essay features and rater preference orderings.
- The study identified essay characteristics influencing personal preferences, distinct from impersonal scoring.
Conclusions:
- Unfolding models offer a robust framework for assessing the quality and underlying processes of rater-mediated assessments.
- Findings have implications for rater training, emphasizing the distinction between objective scoring and subjective preferences.
- The study highlights the importance of considering essay features that may elicit personal preferences in assessment design.
Related Concept Videos
01:23Implicit Personality Theories
214
Implicit personality theory explains how individuals make assumptions about the relationships between personality traits, behaviors, and character types. When people learn that someone possesses a particular trait, they tend to infer the presence of other related characteristics, forming a cohesive impression. This cognitive shortcut plays a crucial role in social interactions and interpersonal judgments.Central Traits and Their InfluenceSolomon Asch's seminal 1946 study highlighted the power...
214
02:16Stereotype Content Model
15.2K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
15.2K
03:00Self-Evaluation: Self-Enhancement and Self-Verification
5.6K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
5.6K
01:24Strategies of Self-Presentation III: Self-Monitoring
129
Self-monitoring is a central construct in understanding individual differences in self-presentation strategies across social contexts. It refers to how individuals observe, regulate, and control their expressive behavior and self-presentation following situational cues. Self-monitoring reflects a person's sensitivity to social appropriateness and willingness to adapt behavior to fit varying interpersonal demands.High vs. Low Self-Monitoring IndividualsIndividuals high in self-monitoring are...
129
01:22Self-Report Tests of Personality
650
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
650
02:55Stereotypes, Prejudice, and Discrimination
94.6K
Humans are very diverse and although we share many similarities, we also have many differences. The social groups we belong to help form our identities (Tajfel, 1974). These differences may be difficult for some people to reconcile, which may lead to prejudice toward people who are different. Prejudice is a negative attitude and feeling toward an individual based solely on one’s membership in a particular social group (Allport, 1954; Brown, 2010). Prejudice is common against people who...
94.6K