Related Experiment Video
Updated: Jun 14, 2025
09:16
Use of a Video Scoring Anchor for Rapid Serial Assessment of Social Communication in Toddlers
Published on: March 14, 2018
10.2K
Are numerical scores important for grant assessment? A cross-sectional study
Ivan Buljan1, David G Pina2, Antonija Mijatović3
1Department of Psychology, Faculty of Humanities and Social Sciences in Split, University of Split, Split, Croatia.
F1000Research
|September 2, 2024
Summary
Removing numerical scores from research proposals did not change reviewer comment language. Linguistic analysis showed consistent writing styles for strengths and weaknesses in both years, indicating no significant impact on reviewer feedback characteristics.
Area of Science:
- Research evaluation
- Scientific funding policy
- Grant proposal assessment
Background:
- Reviewers use quantitative and qualitative criteria for research proposals.
- The European Commission removed numerical scores for Marie Skłodowska-Curie Actions (MSCA) Innovative Training Networks (ITN) in 2020.
- This study investigates linguistic changes in reviewer comments post-score removal.
Purpose of the Study:
- To assess linguistic differences in reviewer comments before and after numerical scoring removal.
- To compare reviewer comment characteristics from 2019 (with scores) and 2020 (without scores).
Main Methods:
- Observational study using MSCA ITN evaluation reports from 2019 and 2020.
- Analysis of individual and consensus comments on Excellence, Impact, and Implementation criteria.
- Linguistic Inquiry and Word Count (LIWC) program used for comment analysis.
Main Results:
- Reviewer comments on proposal strengths showed objectivity, clout, and positive affect.
- Comments on weaknesses were consistently cold and objective.
- Linguistic variables explained only a small fraction (McFadden R²=0.03) of the variance between 2019 and 2020.
Conclusions:
- Removing numerical scores did not alter the linguistic characteristics of reviewer comments.
- Future research should explore qualitative methods to assess content changes in reviewer feedback.
Related Concept Videos
01:10Review and Preview
6.9K
In statistics, several tools are used to interpret the data. Measures of central tendency represent the characteristics of the data, such as mean, median, and mode. Additionally, measures of variance like standard deviation and range are used to find the spread of data from the mean. Relative standing measures the distance between data locations. Commonly used measures of relative standings are percentile, z score, and quartiles.
Percentiles are a type of fractile that partition data into...
Percentiles are a type of fractile that partition data into...
6.9K
00:57Weighted Mean
5.0K
While taking the arithmetic, geometric, or harmonic mean of a sample data set, equal importance is assigned to all the data points. However, all the values may not always be equally important in some data sets. An intrinsic bias might make it more important to give more weightage to specific values over others.
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
5.0K
01:29Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
01:05Introduction to z Scores
360
A z score (or standardized value) is measured in units of the standard deviation. It indicates how many standard deviations the value x is above (to the right of) or below (to the left of) the mean, μ. Values of x that are larger than the mean have positive z scores, and values of x that are smaller than the mean have negative z scores. If x equals the mean, then x has a zero z score. It is important to note that the mean of the z scores is zero, and the standard deviation is one.
z scores...
z scores...
360
02:16Measures of Central Tendency
16.0K
The "center" of a data set is also a way of describing location. The two most widely used measures of the "center" of the data are the mean (average) and the median. The words "mean" and "average" are often used interchangeably. The substitution of one word for the other is common practice. The technical term is "arithmetic mean" and "average" is technically a center location. However, in practice among non-statisticians,...
16.0K
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
124
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
124