Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Enhancing reliability in portfolio assessment: discussions between assessors.

John Pitts1, Colin Coles, Peter Thomas

  • 1NHS Executive South East Region (Wessex Deanery) Winchester, King Alfred's College Winchester, UK. jpitts@chilgrove.u-net.com

Medical Teacher
|July 6, 2002
PubMed
Summary

Assessing portfolios for general practice trainer courses improved reliability. Using paired discussions between assessors significantly boosted the pass/refer judgement accuracy from fair to moderate levels.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Author Correction: Sustainability in medical retina: the environmental impact of using aflibercept 8 mg instead of aflibercept 2 mg in treatment-naïve patients with nAMD.

Eye (London, England)·2026
Same author

Physiotherapy at night: A multicentre prospective audit of respiratory-based physiotherapy services provided at night in Queensland.

Australian critical care : official journal of the Confederation of Australian Critical Care Nurses·2026
Same author

Feasibility of smartphone-based digital phenotyping to measure visual function and mental health outcomes in patients with inherited retinal diseases.

NPJ digital medicine·2026
Same author

Artificial intelligence and clinical informatics in UK ophthalmology training: a national cross-sectional survey.

BMC medical education·2026
Same author

High affinity is insufficient for strong B cell activation by HIV broadly neutralising antibodies.

NPJ vaccines·2026
Same author

Post-operative mobilisation and mobility outcomes after pelvic exenteration surgery for colorectal cancer.

The surgeon : journal of the Royal Colleges of Surgeons of Edinburgh and Ireland·2026

Area of Science:

  • Medical Education
  • General Practice Training
  • Assessment Reliability

Background:

  • Portfolio assessment is crucial for evaluating prospective general practice trainers.
  • Ensuring reliable and consistent assessment is vital for maintaining training standards.
  • Previous individual assessment methods showed fair reliability.

Purpose of the Study:

  • To evaluate the reliability of portfolio assessments for general practice trainer candidates.
  • To compare the reliability of individual assessments versus paired, discussed assessments.
  • To determine if a collaborative assessment approach enhances judgement consistency.

Main Methods:

  • A cohort of participants in a general practice trainer course submitted portfolios.
  • Initial individual assessments were conducted by separate assessors.

Related Experiment Videos

  • Assessors then engaged in open discussions in random pairs to reach composite scores.
  • Kappa statistics were used to analyze inter-rater reliability.

    • Main Results:

    • Individual assessment yielded a kappa statistic of 0.26, indicating fair reliability.
    • Paired discussion between assessors improved the kappa statistic to 0.5, representing moderate reliability.
    • The reliability of global pass/refer judgments showed a notable increase.

    Conclusions:

    • Collaborative assessment through paired discussion enhances the reliability of portfolio evaluations.
    • Implementing paired discussions is a viable strategy to improve the consistency of judgments in general practice trainer assessments.
    • This method offers a more robust approach to certifying prospective general practice trainers.