Predicting response time on self-report rating scale assessments of noncognitive constructs

Area of Science:

• Psychometrics
• Cognitive Psychology
• Behavioral Science

Background:

• Methodological studies highlight response time's predictive power in noncognitive assessments.
• Previous research has examined content trait level and response styles independently.
• The interplay between these factors and potential bias from omitting either is underexplored.

Purpose of the Study:

• To investigate the simultaneous relevance of content trait level and response styles in noncognitive assessments.
• To assess the risk of omitted predictor bias when analyzing response time data.
• To provide a comprehensive model for understanding response time in measurement.

Main Methods:

• Utilized response time data from multiple noncognitive assessments.
• Employed a multilevel regression model to analyze content and response style factors concurrently.
• Compared findings from models that included both factors versus those that omitted one.

Main Results:

• Confirmed the simultaneous relevance of both content trait level and response styles.
• Response style effects were consistently stronger on average but exhibited greater respondent-level variability.
• The multilevel model provided consistent findings supporting the integrated analysis.

Conclusions:

• Accurate noncognitive measurement requires considering both content and response style factors.
• Response styles play a significant role, though their variability necessitates careful interpretation.
• Future research should explore the implications for response time utilization in assessment design.