Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Assessing the validity of computer-based test interpretations: rating reliability and individual differences among

S J Guastello1, M L Rieke, D D Guastello

  • 1Department of Psychology, Marquette University, Milwaukee, WI 53233, USA.

Journal of Personality Assessment
|January 1, 1992
PubMed
Summary

Individual differences in raters affect computer-based test interpretations (CBTIs) accuracy. Reliability of accuracy and usefulness ratings for the Sixteen Personality Factor Questionnaire (16PF) reports varied across studies.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Catastrophe model for the exposure to blood-borne pathogens and other accidents in health care settings.

Accident; analysis and prevention·1999
Same author

Injury analysis and prevention in the developing countries.

Accident; analysis and prevention·1999
Same author

Meta-interpretive reliability of computer-based test interpretations: the Karson Clinical Report.

Journal of personality assessment·1992
Same author

Differentiating symptom clusters of borderline personality disorder.

Journal of clinical psychology·1992
Same author

Estimating age-stratified WAIS-R IQS from scores on the Raven's Standard Progressive Matrices.

Journal of clinical psychology·1991
Same author

The Barnum effect in a computerized Rorschach interpretation system.

The Journal of psychology·1990

Area of Science:

  • Psychological assessment
  • Psychometrics
  • Individual differences research

Background:

  • Computer-based test interpretations (CBTIs) are increasingly used in psychological assessments.
  • Validation studies of CBTIs rely on rater judgments, but the influence of individual rater differences is not fully understood.
  • The Sixteen Personality Factor Questionnaire (16PF) is a widely used personality assessment tool with associated CBTIs.

Purpose of the Study:

  • To investigate the impact of individual rater differences on the accuracy and reliability of computer-based test interpretations (CBTIs).
  • To assess the test-retest reliability of accuracy and usefulness ratings for two specific CBTIs of the Sixteen Personality Factor Questionnaire (16PF).

Main Methods:

  • Three studies were conducted involving student raters evaluating two CBTIs for the 16PF: the Human Resource Development Report (HRDR) and the Narrative Score Report (NSR).

Related Experiment Videos

  • Raters evaluated report accuracy in experimental and nonexperimental contexts.
  • Test-retest reliability was calculated for accuracy and usefulness ratings.

    • Main Results:

    • Specific personality traits (e.g., shrewdness, self-sufficiency, emotional stability) influenced CBTI accuracy ratings, with effects varying by context and report.
    • Test-retest reliability for accuracy ratings was high for both HRDR (.74) and NSR (.81).
    • Test-retest reliability for usefulness ratings was high for HRDR (.75) but moderate for NSR (.54).

    Conclusions:

    • Individual rater characteristics play a role in the perceived accuracy of CBTIs, particularly in experimental settings.
    • The reliability of accuracy ratings for 16PF CBTIs is generally strong, while usefulness ratings show more variability.
    • Findings highlight the importance of considering rater individual differences in the validation of computer-based psychological assessment tools.