Related Experiment Video
Updated: Jul 14, 2026

A Tablet-Based Curriculum-Based Measurement Protocol for Kindergarten Writing
Published on: February 7, 2025
Composition of the panel of reference for concordance tests: do teaching functions have an impact on examinees' ranks
Bernard Charlin1, Robert Gagnon, Evelyne Sauvé
1University of Montreal, Québec, H3C 3J7, Canada. bernard.charlin@umontreal.ca
Background:
Concordance tests are designed to assess the component of uncertainty of clinical reasoning. Scoring is based on a comparison of examinees' answers with those of a panel of reference, including their variability. This allows construction of tests that are close to real clinical life, with all its complexity and ambiguity.
Aim:
This study was carried out to determine the effect of teaching functions of members composing the reference panels on students' scores and ranking.
Methods:
A group of 80 residents in family medicine from a French University (Bobigny) completed a 72-item concordance test. The answers of two panels, each made up of 29 family physicians (teaching function versus non-teaching function), were used to generate the correction keys.
Results:
Correlation between the sets of data obtained with the two panels is high (ICC = 0.98). Concordance scores obtained from the teaching-function panel are higher than scores obtained from the non-teaching-function panel (72.0 versus 76.3; p < 0.001). Ranking provided by the two panels was very similar.
Conclusions:
This legitimizes the use of non-teaching physicians on panels. Panel composition influenced absolute score values: Residents showed more concordance with their academic trainers than with community-based physicians.
Related Concept Videos
Reliability and Validity
Kendall's Coefficient of Concordance
Spearman's Rank Correlation Test
Spearman's test calculates correlation by...
Theory of Attribution II: Kelley's Covariation Theory
Friedman Two-way Analysis of Variance by Ranks
Review and Preview
Percentiles are a type of fractile that partition data into...