在对研究资助申请的同行评审中,排名与评级对比
Robyn Tamblyn1,2,3, Nadyne Girard1, James Hanley2
1Clinical and Health Informatics Research Group, McGill University, Montreal, Canada.
PloS one
|October 5, 2023
概括
研究资助机构应该采用应用程序排名而不是评级. 排名更可靠,不那么有偏见,不受审查员专业知识的影响,改善了卫生研究资助决策.
科学领域:
- 卫生研究资助 卫生研究资助
- 科学资助审查 科学资助审查
背景情况:
- 传统的同行评审评级系统的可靠性和偏见较差.
- 资助机构寻求改进评估研究应用程序的方法.
研究的目的:
- 为了比较应用程序评级与排名的可靠性和偏差.
- 评估审查员特征对评估方法的影响.
主要方法:
- 分析了加拿大卫生研究院提交的3,156份研究申请.
- 评级和排名系统之间的可靠性和偏差指标的比较.
- 评估人口因素 (职业阶段,性别) 和H指的影响.
主要成果:
- 应用程序排名显示了比评级更高的可靠性.
- 排名对审稿人的专业知识和经验不那么敏感.
- 这两种方法都惩罚了职业生涯早期的调查员;排名显示性别偏见较少.
结论:
- 与评级相比,排名提供了对研究应用程序的更可靠,更不偏见的评估.
- 资助机构应该考虑采用排名来提高决策质量.
- 排名减轻了与审查员特征和申请人人口统计学相关的偏见.
相关概念视频
Ranks
246
Unlike parametric methods, nonparametric statistics are ideal for nominal and ordinal data, requiring fewer assumptions about the population's nature or distribution. This makes nonparametric methods easier to apply and interpret, as they do not depend on parameters like mean or standard deviation. One common approach in nonparametric analysis is to sort data according to a specific criterion. For instance, we might arrange weather data from hottest to coldest days in a month or rank cities...
246
Review and Preview
7.6K
In statistics, several tools are used to interpret the data. Measures of central tendency represent the characteristics of the data, such as mean, median, and mode. Additionally, measures of variance like standard deviation and range are used to find the spread of data from the mean. Relative standing measures the distance between data locations. Commonly used measures of relative standings are percentile, z score, and quartiles.
Percentiles are a type of fractile that partition data into...
Percentiles are a type of fractile that partition data into...
7.6K
Friedman Two-way Analysis of Variance by Ranks
213
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
213
Quantifying and Rejecting Outliers: The Grubbs Test
1.6K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.6K
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
137
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
137
Ratio Level of Measurement
18.3K
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated....
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated....
18.3K


