A Bayesian many-facet Rasch model with Markov modeling for rater severity drift

Area of Science:

• Psychometrics
• Statistical Modeling
• Educational Measurement

Background:

• Fair performance assessment necessitates accounting for rater severity.
• The many-facet Rasch model (MFRM) is a standard tool for analyzing rater severity.
• Existing MFRM extensions often overlook temporal dependencies in rater severity (drift).

Purpose of the Study:

• To address the limitations of existing models by incorporating temporal dependency in rater severity.
• To propose a novel Bayesian extension of the MFRM that models rater severity drift.
• To enhance the accuracy of parameter estimation in performance assessments.

Main Methods:

• Developed a Bayesian extension of the MFRM.
• Incorporated time dependency for rater severity parameters using a Markov modeling approach.
• Validated the model through simulation experiments and real-world data application.

Main Results:

• The proposed Bayesian MFRM effectively models time-dependent rater severity.
• Improved estimation accuracy for time-specific rater severity parameters was achieved.
• Enhanced accuracy in estimating other rater parameters and improved overall model fitting.

Conclusions:

• The Bayesian MFRM with Markov modeling offers a more accurate approach to performance assessment by accounting for rater severity drift.
• This method provides more reliable insights into rater behavior over time.
• The model demonstrates practical utility in both simulated and actual data scenarios.