Is the Factor Observed in Investigations on the Item-Position Effect Actually the Difficulty Factor?

Area of Science:

• Psychometrics
• Educational Measurement

Background:

• Confirmatory factor analysis models for difficulty and item-position effects are similar.
• This similarity hinders the dissociation of these two distinct factors in test data analysis.

Purpose of the Study:

• To investigate a statistical method for discriminating between the difficulty factor and the item-position effect.
• To determine if the item-position effect can be identified independently of item difficulties.

Main Methods:

• Two studies were conducted: one with Advanced Progressive Matrices (APM) data and another with simulated data.
• Statistical ex post manipulation of item difficulties was employed to isolate factors.
• Simulated data was manipulated to minimize the influence of the difficulty factor.

Main Results:

• The item-position effect was successfully detected in the APM dataset.
• In simulated datasets where the range of item difficulties was reduced to zero, the item-position factor was still observed.
• These findings support the theoretical independence of the item-position effect from item difficulties.

Conclusions:

• The statistical manipulation of item difficulties is a viable method for dissociating the item-position effect from the difficulty factor.
• The item-position effect is a robust phenomenon in psychometric analysis, detectable even when item difficulty variations are minimized.