Related Experiment Video
Updated: Dec 31, 2025
08:27
Applying an eMASS Customization Program as a Research Tool to Evaluate Consumer Benefits
Published on: September 27, 2019
7.2K
Indices of Subscore Utility for Individuals and Subgroups Based on Multivariate Generalizability Theory.
Mark R Raymond1, Zhehan Jiang2
1National Board of Medical Examiners, Philadelphia, PA, USA.
Educational and Psychological Measurement
|January 15, 2020
Summary
New indices enhance score profile utility evaluation beyond traditional methods. These conditional indices, based on generalizability theory, offer more precise reliability insights for individual examinees.
Area of Science:
- Psychometrics
- Statistical Modeling
- Educational Measurement
Background:
- Traditional subscore utility evaluation uses reliability indices and correlations.
- Correlational methods overlook variations in subtest means.
- Existing group-level reliability indices like do not capture individual score profile reliability.
Purpose of the Study:
- Propose two conditional indices for score profile utility grounded in multivariate generalizability theory.
- Address the limitation of group-average reliability by providing individual-level insights.
- Enhance the evaluation of subscore utility beyond traditional correlational methods.
Main Methods:
- Developed two conditional indices based on generalizability theory.
- Index 1: Ratio of observed profile variance to random error variance.
- Index 2: Proportion of observed variability attributable to true score variance.
- Evaluated indices using two empirical examples and simulation data.
Main Results:
- Proposed indices provide consistent estimates of profile error variance.
- Indices offer information beyond traditional subscore utility measures.
- Simulation indicated potential for artificially large index values in 5-8% of cases.
Conclusions:
- Conditional indices offer a more nuanced understanding of score profile utility.
- These indices provide valuable information for individual examinees and subgroups.
- Further research should explore applications and refine the indices.
Related Concept Videos
01:19Expected Frequencies in Goodness-of-Fit Tests
6.8K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
6.8K
01:02Quantifying and Rejecting Outliers: The Grubbs Test
3.4K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
3.4K
01:21Friedman Two-way Analysis of Variance by Ranks
451
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
451
01:11Variability: Analysis
390
Measures of variability are statistical metrics that reveal the dispersion pattern within a dataset. They are pivotal in biostatistics, providing insights into the heterogeneity within health and biological data. Variability signifies the degree to which data points diverge from one another, helping researchers understand the potential range of values and associated uncertainty within the data.
The range is a simple measure of variability, indicating the difference between the highest and...
The range is a simple measure of variability, indicating the difference between the highest and...
390
01:20Comparing the Survival Analysis of Two or More Groups
505
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
505
01:28Impact of Groups on Individuals
260
Groups play a fundamental role in shaping individual behavior, as they establish norms that guide interactions and decision-making. Social psychology examines how individuals conform to group expectations, often adjusting their attitudes and actions to align with group norms. These norms can be formal, such as workplace policies, or informal, such as unspoken social expectations within a fraternity.Conformity and Social InfluenceConformity arises when individuals modify their behaviors or...
260