Related Experiment Video
Updated: Jul 12, 2026
10:26
Problem-Solving Before Instruction (PS-I): A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
The influence of equating methodology on reported trends in PISA
Eveline Gebhardt1, Raymond J Adams
1Australian Council for Educational Research, 19 Prospect Hill Rd., Camberwell, VIC 3124, Australia. gebhardt@acer.edu.au
Journal of Applied Measurement
|September 7, 2007
Summary
This study examines how different statistical methods impact Programme for International Student Assessment (PISA) trend analyses between 2000 and 2003. Results show that the choice of equating methodology and regression models significantly alters trend conclusions.
Area of Science:
- Educational measurement
- Comparative education research
Background:
- The Programme for International Student Assessment (PISA) published trend indicators in 2005, comparing 2000 and 2003 results.
- PISA utilized Rasch Modelling for test equating, involving linear transformations to map 2003 scores to 2000 scores.
Purpose of the Study:
- To investigate the sensitivity of PISA trend analyses to different test equating methodologies, regression models, and linking items.
- To compare PISA's standard trend analysis approach with an alternative joint Rasch scaling method.
Main Methods:
- Comparison of PISA's linear transformation equating method with a joint Rasch scaling approach where item parameters are estimated separately for each country.
- Analysis of marginal mean score changes versus conditional mean score changes adjusted for background characteristics.
Main Results:
- The choice of equating methodology and regression models influences the outcomes of trend analyses.
- Using conditional means instead of marginal means leads to different conclusions regarding trends at both country and within-country levels.
Conclusions:
- Trend indicators derived from PISA data are sensitive to the statistical methodologies employed.
- The reporting of trend data should consider the impact of equating methods and the use of conditional versus marginal means for more accurate interpretations.
Related Concept Videos
01:52Regression Toward the Mean
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when researchers try to extrapolate results...
01:09Decision Making: P-value Method
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can have a...
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can have a...
01:29Bias in Epidemiological Studies
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
01:15One-Way ANOVA: Equal Sample Sizes
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
00:54Ratio Level of Measurement
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated. For...
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated. For...
01:24Testing a Claim about Population Proportion
A complete procedure for testing a claim about a population proportion is provided here.
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...