Confidence in mathematics is confounded by responses to reverse-coded items

Area of Science:

• Educational Measurement
• Psychometrics
• Survey Methodology

Background:

• Confidence in mathematics is a key factor in student achievement.
• Reverse-coded items are often used in surveys to enhance measurement quality.
• Potential confounding effects of reverse-coded items require investigation.

Purpose of the Study:

• To investigate the impact of reverse-coded items on measuring mathematics confidence.
• To identify subgroups of students affected by reverse-coded items.
• To assess the effect on psychometric properties like reliability and validity.

Main Methods:

• Utilized data from the 2019 Trends in International Mathematics and Science Study (TIMSS).
• Applied mixture modeling to identify response patterns to reverse-coded items in a Saudi Arabian cohort (n=4,515).
• Employed person fit indicators (Lz*, U3, Guttman errors) for verification.

Main Results:

• Identified two subgroups (approx. 9% of sample) with inconsistent responses to reverse-coded items.
• Psychometric analyses showed improved internal consistency reliability and factorial validity after removing aberrant responders.
• Reverse-coded items were associated with distorted item parameters and measurement error.

Conclusions:

• Reverse-coded items can introduce systematic measurement error, compromising scale reliability and validity.
• The findings highlight the need to reconsider the use of reverse-coded items in surveys.
• Particular caution is advised for younger populations and students with lower academic achievement.