Interobserver variation in the AO/OTA fracture classification system for pilon fractures: is there a problem?

To evaluate the interobserver variation for the AO/OTA fracture classification system: region forty-three-pilon fractures.

One senior attending, two fellows (one trauma, one foot and ankle), one junior orthopaedic resident, and one experienced research coordinator independently classified eighty-four sets of radiographs. The evaluator was blinded as to treatment and functional outcome. The radiographs initially used to manage the patients were evaluated; no special radiographs or standardized radiographic techniques were used. The kappa statistic, Williams index, and SAV statistic were calculated.

Using the SAV statistic to quantify rater agreement beyond that expected by chance alone, the average chance-adjusted agreement among the raters was 0.57 for fracture type, 0.43 for group, and 0.41 for subgroup. This is equivalent to moderate agreement (0.41 to 0.60). The kappa statistic was used to determine whether there was difficulty with any specific category of the AO type classification among raters for selecting fracture type (A, B, C). Kappa values were 0.49 for type A, 0.58 for type B, 0.57 for type C, all of which were considered adequate.

These data are similar to others reported for interobserver agreement with the AO/OTA fracture classification and other classification systems. The issue of individual judgement in taking a continuous variable (fracture pattern) and compartmentalizing it into a dichotomous variable (fracture classification system) is highlighted by these data. Determination of fracture types alone (type A, B, or C) would seem to be sufficient for clinical research where fracture severity should be reported as a variable.