Detecting random responders with infrequency scales using an error-balancing threshold.

Area of Science:

• Psychological Measurement
• Quantitative Psychology
• Research Methodology

Background:

• Infrequency scales are widely used for data screening in research due to their accessibility.
• Concerns exist regarding the interpretation and application of infrequency items, with a lack of empirical guidelines for cutoff implementation.
• Previous research suggests infrequency item functioning may be more complex than initially assumed.

Purpose of the Study:

• To compare the effectiveness of a zero-tolerance threshold versus an error-balancing threshold for detecting random responding using infrequency items.
• To provide empirically based guidelines for implementing cutoffs in infrequency scales.
• To evaluate the potential for overly stringent cutoffs to exclude valid data.

Main Methods:

• Comparison of two distinct methods for detecting random responding: a strict zero-tolerance threshold and a threshold balancing classification error rates.
• Utilized infrequency items as the primary tool for data screening.
• Analysis focused on the characteristics of data screened by each method.

Main Results:

• The zero-tolerance approach screened data less indicative of careless responding compared to the error-balancing approach.
• The error-balancing threshold demonstrated superior performance in identifying genuinely random responses.
• Traditional zero-tolerance cutoffs may excessively exclude participants with valid data.

Conclusions:

• The standard "zero-tolerance" approach to infrequency scales might be overly stringent, leading to the removal of meaningful data.
• An error-balancing threshold is recommended for more accurate detection of careless responding.
• Future research should focus on refining cutoff guidelines for infrequency scales to optimize data screening and preserve valid responses.