Careless Responding May Reduce Data Quality in Self-Report Questionnaire Research: Evidence From Adult Temperament...

Area of Science:

• Psychology
• Research Methodology
• Data Quality

Background:

• Online self-report studies are increasing, raising concerns about data quality.
• Response bias is a significant threat to the validity of online survey data.

Purpose of the Study:

• To determine the prevalence of response bias in a cross-national sample.
• To evaluate the impact of different types of response bias on psychometric outcomes.
• To assess the effectiveness of post-hoc data screening methods.

Main Methods:

• Analyzed data from 1,497 adult participants in the US and Lithuania.
• Identified indicators of careless responding (invariant/random patterns) and clinical threats (social desirability, discrepancy ratings).
• Evaluated the impact of including/excluding biased data on internal consistency and factor structure.

Main Results:

• Excluding careless responding improved psychometric indicators (internal consistency, factor structure).
• Excluding data flagged by clinical controls decreased psychometric quality.
• Forced answering in surveys correlated with higher rates of careless responding.

Conclusions:

• Mitigating response bias is essential for reliable online self-report research.
• Post-hoc screening of both careless responding and clinical threats is valuable for enhancing data quality.
• Findings raise concerns about the integrity of existing survey-based datasets.