在开放式测试项目的手动和自动分数的联合项目响应模型
Daniel Bengs1,2, Ulf Brefeld2, Ulf Kroehne1
1Leibniz Institute for Research and Information in Education, Frankfurt, Germany.
Psychometrika
|June 16, 2025
概括
开放式测试项目的自动评分提高了效率,但引入了错误. 新的联合模型通过计算这些自动评分错误来准确估计学生的能力,从而提高了教育测量.
科学领域:
- 教育测量教育的测量
- 心理测量 心理测量 心理测量
- 教育中的人工智能
背景情况:
- 开放式测试项目提高了构造有效性,但需要昂贵的手动评分.
- 手动评分限制了在适应性测试中使用项目数据的使用.
- 使用机器学习的自动评分提供了效率,但引入了分类错误.
研究的目的:
- 开发统计模型,将手动和自动分数整合到开放项目的数据中.
- 为了考虑到测量模型中的自动评分固有的分类错误.
- 提高教育评估中的能力估计的准确性.
主要方法:
- 提出了两种联合项目响应理论 (IRT) 模型,包括手动和自动分数.
- 扩展现有的IRT模型,包括一个自动评分错误的组件.
- 评估模型使用2012年国际学生评估计划 (PISA) 的数据和模拟数据集.
主要成果:
- 拟议的联合模型有效地减轻了分类错误对能力估计的影响.
- 与基线模型相比,证明了更好的准确性,该模型忽略了自动评分错误.
- 在真实世界 (PISA) 和模拟数据上验证了模型的性能.
结论:
- 联合建模提供了一个强大的方法来处理在教育测试中自动得分的开放式项目.
- 当使用自动评分时,对分类错误的核算对于准确的能力估计至关重要.
- 这些模型在适应性和大规模评估中提高了开放项目的有效性和实用性.
更多相关视频
07:43Author Spotlight: A Novel Setup to Conduct Naturalistic Laboratory Experiments with Real Human Actors in Scenarios
Published on: August 4, 2023
2.2K
06:48Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
Published on: June 25, 2019
9.3K
相关概念视频
Response Surface Methodology
288
Response Surface Methodology (RSM) is a collection of statistical and mathematical techniques used to develop, improve, and optimize processes. It is particularly valuable when many input variables or factors potentially influence a response variable.
The process of RSM involves several key steps:
The process of RSM involves several key steps:
288
Self-Report Tests of Personality
467
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
467
Multiple Regression
3.2K
Multiple regression assesses a linear relationship between one response or dependent variable and two or more independent variables. It has many practical applications.
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
3.2K
Mechanistic Models: Compartment Models in Individual and Population Analysis
89
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
89
Goodness-of-Fit Test
4.1K
The goodness-of-fit test is a type of hypothesis test which determines whether the data "fits" a particular distribution. For example, one may suspect that some anonymous data may fit a binomial distribution. A chi-square test (meaning the distribution for the hypothesis test is chi-square) can be used to determine if there is a fit. The null and alternative hypotheses may be written in sentences or stated as equations or inequalities. The test statistic for a goodness-of-fit test is given as...
4.1K
Measures of Intelligence
7.9K
Psychologists measure intelligence by using standardized tests that produce a score known as the intelligence quotient or IQ. To understand IQ tests, it's important to recognize the key principles behind their construction: validity, reliability, and standardization.
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
7.9K
