Inability of rheumatologists to describe their true policies for assessing rheumatoid arthritis

Eighty nine British and Australian rheumatologists took part in a study to discover how accurately they could describe their procedures for measuring disease severity in rheumatoid arthritis. The relative importance they attached to different clinical and laboratory variables showed a very wide variation, and these stated policies were generally poor at predicting their actual judgments when assessing 'paper patients' (r2 = 39%). Policies based on equal weighting of all variables, while also poor predictors (r2 = 41%), were nevertheless superior to their stated policies for 49 respondents. Policies calculated by judgment (linear regression) analysis were much more successful predictors (R2 = 73%). Unhurried, detailed interviews with four experienced rheumatologists provided carefully considered statements of assessment policy, but these also were poor predictors of routine assessments of outpatients (r2 = 34%) compared with policies calculated by clinical judgment analysis, even when these were applied to new data (R2 = 88%).