基于工作场所的评估有多好地支持总结性委托决策? 一个多机构的可通用性研究.
Michael S Ryan1,2, Katherine A Gielissen3, Dongho Shin4
1Department of Pediatrics, University of Virginia, Charlottesville, Virginia, USA.
Medical education
|January 3, 2024
概括
基于工作场所的评估 (WBAs) 显示在多个机构中可靠性很低. 诸如教师发展和规模选择等因素并没有持续地提高WBA对高风险决策的可靠性.
科学领域:
- 医学教育 医学教育
- 卫生职业教育中的评估
背景情况:
- 基于工作场所的评估 (WBA) 对于评估入住资格的核心可信任专业活动 (CEPAER) 是至关重要的.
- 之前的单一机构研究报告了WBAs在衡量向委托的进展方面不一致的可靠性.
- 教师的发展,评分员的参与和规模的选择是影响WBA可靠性的潜在因素.
研究的目的:
- 进行一个多机构的可通用性研究.
- 确定特定因素对WBA可靠性的影响.
主要方法:
- 分析了7所机构 (2018-2020年) 的文职水平学生的53965份WBA.
- 应用可概括性理论和一个交叉分类的随机效应模型来分解方差元件.
- 评估包括指定的评估员 (DA),规模和可信的专业活动 (EPA) 在可靠性方面的因素,以phi系数>0.7作为值.
主要成果:
- 在76%的分析中观察到归因于学习者的低差异 (<10%),因此需要大量观察以确保可靠性.
- 可靠性差异很大,平均需要60次观察.
- 诸如DA,规模或EPA等因素与改善WBA可靠性没有一致的联系.
结论:
- 跨多个站点的WBA显示出相对较低的可靠性.
- 比以前想象的,WBA的概括性可能不受教师发展,评分员参与或规模选择的影响.
- 虽然WBA对于形成性反很有用,但对于高风险的总结性委托决策,WBA可能缺乏一致的可靠性.
相关概念视频
Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
Surveys
14.8K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
14.8K
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
129
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
129
Measures of Intelligence
7.3K
Psychologists measure intelligence by using standardized tests that produce a score known as the intelligence quotient or IQ. To understand IQ tests, it's important to recognize the key principles behind their construction: validity, reliability, and standardization.
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
7.3K
Case Studies
11.7K
There are many research methods available to psychologists in their efforts to understand, describe, and explain behavior and the cognitive and biological processes that underlie it.
11.7K


