Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Counting on citations: a flawed way to measure quality.

Garry Walter1, Sidney Bloch, Glenn Hunt

  • 1Rivendell Unit, Child Adolescent and Family Psychiatric Services, Thomas Walker Hospital, Concord West, NSW, Australia. gwalter@mail.usyd.edu.au

The Medical Journal of Australia
|March 14, 2003
PubMed
Summary

Journal Impact Factor and citation counts are poor measures of scientific quality. A new, replicable system is introduced to identify the best scientific articles by reading them.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Facilitating the ecosystem-based management transition in Aotearoa New Zealand.

Journal of the Royal Society of New Zealand·2025
Same author

The Blue Ribbon Committee II Report and Recommendations on Surgical Education and Training in the United States: 2024.

Annals of surgery·2024
Same author

Goals, Structure, and Financing of Surgical Residency Training: A Subcommittee Report of the Blue Ribbon Committee II.

Annals of surgery·2024
Same author

Position statements of the Royal Australian and New Zealand College of Psychiatrists: An historical perspective.

Australasian psychiatry : bulletin of Royal Australian and New Zealand College of Psychiatrists·2023
Same author

Understanding attributes and characteristics of the PA leader-physician relationship.

JAAPA : official journal of the American Academy of Physician Assistants·2022
Same author

Codes of ethics for psychiatrists: past, present and prospect.

Psychological medicine·2022

Area of Science:

  • Bibliometrics
  • Scientific Publishing
  • Research Evaluation

Background:

  • Journal Impact Factor and citation counts are widely used but flawed metrics for assessing scientific quality.
  • Over-reliance on these quantitative measures can misrepresent the true value of research articles.

Purpose of the Study:

  • To critically evaluate the limitations of current bibliometric indicators in scientific quality assessment.
  • To introduce a practical and replicable system for identifying high-quality journal articles through direct evaluation.

Main Methods:

  • Development of a novel system for article quality assessment.
  • Focus on qualitative review of individual articles rather than solely relying on journal-level metrics.
  • Ensuring the system is easily replicable by other researchers and institutions.

Related Experiment Videos

Main Results:

  • Demonstration of a feasible method for identifying top-tier articles within a journal.
  • Highlighting the inadequacy of Impact Factor and citation counts as sole determinants of scientific merit.
  • Providing a framework for more accurate research quality appraisal.

Conclusions:

  • Direct reading and qualitative assessment are essential for judging scientific article quality.
  • The proposed system offers a superior alternative to traditional bibliometric measures for identifying impactful research.
  • Encouraging the adoption of more nuanced methods in research evaluation and scientific publishing.