Unfolding the Network of Peer Grades: A Latent Variable Approach

Area of Science:

• Educational Technology
• Statistical Modeling
• Bayesian Inference

Background:

• Peer grading is widely used in Massive Open Online Courses (MOOCs) and traditional classrooms.
• It reduces instructor workload and enhances student learning through active engagement.
• Peer grading data exhibit complex dependencies due to the network structure of interactions.

Purpose of the Study:

• To develop a statistical framework for analyzing complex peer grading data.
• To address biases in aggregated grades caused by unmodeled grader effects.
• To provide a method for assessing individual grader performance and understanding grading networks.

Main Methods:

• Introduction of a latent variable model framework for peer grading data analysis.
• Development of a fully Bayesian procedure for statistical inference.
• Application of the model to two real-world peer grading datasets.

Main Results:

• The proposed model yields more accurate aggregated grades by accounting for heterogeneous grading behaviors using latent variables.
• It enables the assessment of individual student grader performance, identifying reliable graders.
• The Bayesian approach facilitates straightforward uncertainty quantification for model parameters and latent variables.

Conclusions:

• The latent variable model offers a robust approach to analyzing peer grading data, improving grade accuracy and providing valuable insights.
• This method enhances the understanding of grader reliability and student performance dynamics within peer assessment systems.
• The Bayesian framework ensures reliable inference and uncertainty quantification, making it suitable for educational data analysis.