Predictive validity of the Global Assessment Form used in a final-year undergraduate rotation in emergency medicine

To determine whether the predictive validity of Global Assessment Form (GAF) knowledge subdomain marks exceeds that of the overall GAF marks with respect to written examination marks for an undergraduate rotation in emergency medicine, and to determine the interdependence between subdomain marks on the GAF.

Final-year clinical clerks completing a four-week rotation through the emergency departments of a university teaching center were evaluated using both a ten-subdomain GAF for clinical performance and an independent written examination. The GAF and examination marks were prospectively obtained for clinical clerks over a two-year period. Pearson correlations were calculated between examination marks and both the GAF knowledge subdomain and the GAF overall mark. Olkin's Z score was calculated to determine the significance of the difference between correlations. Interdependencies between subdomains of the GAF were calculated using an alpha coefficient and inter-item correlations.

Data sets were reviewed for 347 clinical clerks. Nine sets of data were excluded (incomplete evaluations); 338 sets were analyzed. Means for overall clinical mark and examination mark were 80.11% (SD = 4.375) and 81 (SD = 7.66). Among subdomains, knowledge had the highest correlation with the examination mark (0.19). Overall clinical marks had lower correlation with the examination marks (0.169); the difference was not significant (Olkin's Z = 0.40). The correlation of the examination marks with the average marks of all subdomains excluding knowledge was even lower (0.12). The tenitem alpha for the GAF was 0.92.

Clinical GAF assessments of knowledge, as measured by written examination, do not appear to be any more predictive than overall clinical impression. There is substantial consistency between subdomain scores, suggesting that assessors are not effectively discriminating between them when assigning marks.