Related Experiment Video
Updated: Aug 15, 2025
10:58
Multimedia Battery for Assessment of Cognitive and Basic Skills in Mathematics BM-PROMA
Published on: August 28, 2021
4.6K
The raters' differences in Arabic writing rubrics through the Many-Facet Rasch measurement model.
Harun Baharudin1, Zunita Mohamad Maskor2, Mohd Effendi Ewan Mohd Matore3
1Faculty of Education, Center of Diversity and Education, Universiti Kebangsaan Malaysia (UKM), Selangor, Malaysia.
Frontiers in Psychology
|January 2, 2023
Summary
Writing assessment reliability and validity improve by modeling rater differences using the Many-Facet Rasch measurement model (MFRM). This approach addresses leniency variations among raters, enhancing score accuracy.
Area of Science:
- Educational Measurement
- Psychometrics
- Writing Assessment
Background:
- Writing assessment quality depends on rubric, task, and rater consistency.
- Systematic analysis of rater differences is often overlooked in rubric evaluation.
- Analytic rubrics are used to assess writing quality across various domains.
Purpose of the Study:
- To examine rater differences in writing assessment.
- To evaluate the impact of rater variations on rubric reliability and validity.
- To apply the Many-Facet Rasch measurement model (MFRM) for modeling rater effects.
Main Methods:
- Utilized the Many-Facet Rasch measurement model (MFRM).
- Analyzed rating data from 15 Arabic essays evaluated by five raters.
- Assessed rating quality across four domains: vocabulary, grammar, language use, and organization.
Main Results:
- Significant leniency variations were observed among raters, necessitating MFRM modeling.
- Modeling rater differences reduced measurement error and improved criterion validity.
- MFRM adjustments enhanced score correlations, indicating increased reliability.
Conclusions:
- Rater differences significantly impact writing assessment reliability and validity.
- The Many-Facet Rasch measurement model (MFRM) is effective for modeling rater variations.
- MFRM enhances the evaluation of analytic writing rubrics and overall assessment quality.
Keywords:
Arabic essays validationMany-Facet Rasch measurement model (MFRM)analytic rubricraterswriting assessmentwriting domainsMore Related Videos
15:00A Tablet-Based Curriculum-Based Measurement Protocol for Kindergarten Writing
Published on: February 7, 2025
701
09:00Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education
Published on: August 16, 2024
850
Related Concept Videos
00:54Ratio Level of Measurement
18.7K
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated....
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated....
18.7K
01:16Response Surface Methodology
213
Response Surface Methodology (RSM) is a collection of statistical and mathematical techniques used to develop, improve, and optimize processes. It is particularly valuable when many input variables or factors potentially influence a response variable.
The process of RSM involves several key steps:
The process of RSM involves several key steps:
213
02:16Surveys
14.9K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
14.9K
01:10Review and Preview
7.7K
In statistics, several tools are used to interpret the data. Measures of central tendency represent the characteristics of the data, such as mean, median, and mode. Additionally, measures of variance like standard deviation and range are used to find the spread of data from the mean. Relative standing measures the distance between data locations. Commonly used measures of relative standings are percentile, z score, and quartiles.
Percentiles are a type of fractile that partition data into...
Percentiles are a type of fractile that partition data into...
7.7K
01:20Spearman's Rank Correlation Test
937
Spearman's rank correlation test, also known as Spearman's rho, is a nonparametric method for assessing the strength and direction of association between two variables. This test is particularly valuable when the data distribution is unknown or when the assumption of normality does not hold. Named after the English psychologist and statistician Dr. Charles Edward Spearman, it serves as the nonparametric counterpart to Pearson's correlation coefficient.
Spearman's test calculates...
Spearman's test calculates...
937
01:52Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K