Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Rating intervals: an experiment in peer review.

J G Green1, F Calhoun, L Nierzwicki

  • 1Division of Research Grants, National Institutes of Health, Bethesda, Maryland 20892.

FASEB Journal : Official Publication of the Federation of American Societies for Experimental Biology
|June 1, 1989
PubMed
Summary

National Institute of Health (NIH) grant scoring became more uniform. An experiment standardizing rating scales and voting intervals improved score distribution, aiding funding decisions.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Anaemia is independently associated with mortality in patients with hepatocellular carcinoma.

ESMO open·2024
Same author

Search for Gamma-Ray Spectral Lines from Dark Matter Annihilation up to 100 TeV toward the Galactic Center with MAGIC.

Physical review letters·2023
Same author

Fiscal and monetary policies: the cutting edge of advocacy and research on population health and climate change.

Perspectives in public health·2021
Same author

A risk algorithm that predicts alcohol use disorders among college students.

European child & adolescent psychiatry·2021
Same author

[Major depressive episode in college freshmen: prevalence, academic functioning and receipt of treatment].

Tijdschrift voor psychiatrie·2021
Same author

Predicting the incidence of non-suicidal self-injury in college students.

European psychiatry : the journal of the Association of European Psychiatrists·2019

Area of Science:

  • Health Services Research
  • Biomedical Research Policy

Background:

  • The National Institute of Health (NIH) peer review system faces challenges with skewed priority score distributions.
  • Variable rating behaviors among study sections complicate NIH grant funding decisions.

Purpose of the Study:

  • To enhance uniformity in rating behavior across NIH study sections.
  • To achieve a broader and more consistent description of scores in grant evaluations.

Main Methods:

  • An experiment involving 24 NIH study sections was conducted.
  • Study sections were randomly assigned to use either 0.1 or 0.5 intervals for voting.
  • Standard adjectival descriptors and rating scales were implemented.

Main Results:

Related Experiment Videos

  • The experiment led to greater uniformity in rating behavior.
  • Score distributions became broader and more consistent across study sections.
  • The implemented changes have been adopted into standard NIH practice.

    • Conclusions:

    • Standardizing rating scales and voting intervals effectively addresses variability in NIH peer review.
    • The revised scoring system improves the reliability of grant application evaluations.
    • These changes support more equitable and efficient allocation of research funding.