Post-hoc Rasch analysis of optimal categorization of an ordered-response scale

Area of Science:

• Psychometrics
• Educational Psychology

Background:

• Self-efficacy scales are crucial for measuring confidence in psychomotor skills.
• Ordered-response scales require careful category construction for accurate measurement.
• Conventional statistical methods may not adequately assess scale categorization.

Purpose of the Study:

• To determine the optimal categorization of a self-efficacy ordered-response scale.
• To compare Rasch analysis statistics and parameter estimates with conventional statistics.
• To evaluate the utility of Rasch analysis for refining ordered-response scales.

Main Methods:

• Rasch rating scale model applied to a 50-item psychomotor self-efficacy scale administered to 2,022 children.
• Original five categories collapsed into two, three, and four categories, creating 14 datasets for analysis.
• Comparison of Rasch model-data fit, category, and separation statistics with conventional statistics (e.g., coefficient alpha).

Main Results:

• The optimal categorization for the self-efficacy scale was found to be a three-category construct, differing from the original five.
• Rasch threshold estimates effectively determined the order of categorization.
• Item separation statistics aided in confirming optimal categorization once the order was established.
• Coefficient alpha was not helpful in determining optimal categorization.

Conclusions:

• Rasch analysis is a valuable post-hoc method for optimizing the categorization of ordered-response scales.
• Rasch statistics provide more sensitive and informative measures for scale refinement than conventional methods.
• The study highlights the importance of appropriate category structure in self-efficacy measurement.