在心理测试中使用可解释机器学习来检测差异物品的功能
Elisabeth Barbara Kraus1, Johannes Wild2, Sven Hilbert2
1LMU Munich, Germany.
Applied psychological measurement
|July 26, 2024
概括
本研究引入了一种新的机器学习方法来检测不公平的测试项目,提供与现有技术相比可靠的结果. 该方法有效地识别了教育评估中的偏见,确保对所有考生进行更公平的评估.
科学领域:
- 心理测量 心理测量 心理测量
- 机器学习 机器学习
- 教育测量教育的测量
背景情况:
- 测试不公平性源于人口统计学对心理测量模型残余差异的影响.
- 现有的方法在响应模式和人口统计学属性之间的复杂关系中扎.
研究的目的:
- 介绍一种结合心理测量和机器学习的新方法,用于调查测试公平性.
- 测量测试项和潜在能力分数对人口特征的预测重要性.
主要方法:
- 利用随机森林从测试答案预测人口统计学属性.
- 进行模拟研究以评估各种条件下的方法性能.
- 将该方法应用于小学阅读理解测试,以预测移民背景.
主要成果:
- 这种新方法可靠地检测到不公平的项目,与Mantle-Haenszel统计和后勤回归相似.
- 这种方法可以有效地通用到多维尺度.
- 阅读理解测试中的一个项目根据拟议的标准被确定为不公平.
结论:
- 提出的基于机器学习的方法提供了一个强大的方法来识别测试不公平.
- 这种技术增强了在教育评估中检测差异性项目功能的能力.
- 研究结果支持使用机器学习来提高标准化测试的公平性和有效性.
相关概念视频
01:22Self-Report Tests of Personality
336
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
336
01:29Measures of Intelligence
7.1K
Psychologists measure intelligence by using standardized tests that produce a score known as the intelligence quotient or IQ. To understand IQ tests, it's important to recognize the key principles behind their construction: validity, reliability, and standardization.
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
7.1K
01:09Statistical Methods to Analyze Parametric Data: Student t-Test and Goodness-of-Fit Test
1.6K
In parametric statistics, two fundamental tests stand out for their utility and wide application: the Student's t-test and goodness-of-fit tests. These tests provide researchers with a robust method for drawing insights from data, testing hypotheses, and making informed decisions based on their findings.
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with...
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with...
1.6K
01:08Fisher's Exact Test
437
Fisher's exact test is a statistical significance test widely used to analyze 2x2 contingency tables, particularly in situations where sample sizes are small. Unlike the chi-squared test, which approximates P-values and assumes minimum expected frequencies of at least five in each cell, Fisher's exact test calculates the exact probability (P-value) of observing the data or more extreme results under the null hypothesis. This feature makes it especially valuable when the assumptions of...
437
01:19Expected Frequencies in Goodness-of-Fit Tests
2.5K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
2.5K
00:57Behrens–Fisher Test
74
The Behrens-Fisher test is a statistical method designed to address the Behrens-Fisher problem, which arises when comparing the means of two normally distributed populations with unequal variances. Unlike the Student's t-test, which assumes equal variances, the Behrens-Fisher test allows for mean comparison without this restrictive assumption. This flexibility makes it particularly valuable in scenarios where two independent samples exhibit normality but lack variance homogeneity.
This test...
This test...
74