Is there more than one approach to evaluating the variability of surgeons' performance?

To demonstrate that the variability found to be significant between surgeons' performances within a small group does not necessarily mean that this significance applies to the entire field of that specific type of surgery. It is common for inferences and recommendations for an entire field to be based on the variability within a small group of surgeons. The variability between groups usually remains unknown.

An analysis of variance was used to assess the statistical significance of the variability among surgeons' performances of a specific type of surgery within the studied sample. The intraclass correlation coefficient was used to investigate how large a segment of this variability can be explained by a surgeon-related factor for the entire surgeon population of a specific field. The topic was illustrated using data obtained from a group of seven surgeons who operated on the penetrating rotator cuff tears of 742 patients.

There were statistically significant differences between seven surgeons in the improvement of pain and the range of shoulder joint motion. However, only a small (≤2%) and statistically non-significant part of this variability could be explained by a difference between surgeons when the results were interpolated across the entire population of shoulder surgeons.

Variability in performance within a group of surgeons performing a specific type of surgery cannot be generalized to include the performance of all surgeons doing the same type of surgery without additional statistical analyses.