Related Experiment Video
Updated: Apr 15, 2026
Surgical Trunk Oriented Laparoscopic Right Hemicolectomy (ST-LRH) for Right-Sided Colon Cancer
Published on: July 25, 2025
Reliability of surgeon-specific reporting of complications after colectomy
Terry Shih1, Adam I Cole, Paul M Al-Attar
1*Department of Surgery, Academic Surgeon Development Program †Michigan Surgery Quality Collaborative, University of Michigan, Ann Arbor, MI.
Objective:
We sought to determine the reliability of surgeon-specific postoperative complication rates after colectomy.
Background:
Conventional measures of surgeon-specific performance fail to acknowledge variation attributed to statistical noise, risking unreliable assessment of quality.
Methods:
We examined all patients who underwent segmental colectomy with anastomosis from 2008 through 2010 participating in the Michigan Surgical Quality Collaborative Colectomy Project. Surgeon-specific complication rates were risk-adjusted according to patient characteristics with multiple logistic regression. Hierarchical modeling techniques were used to determine the reliability of surgeon-specific risk-adjusted complication rates. We then adjusted these rates for reliability. To evaluate the extent to which surgeon-level variation was reduced, surgeons were placed into quartiles based on performance and complication rates were compared before and after reliability adjustment.
Results:
A total of 5033 patients (n = 345 surgeons) undergoing partial colectomy reported a risk-adjusted complication rate of 24.5%. Approximately 86% of the variability of complication rates across surgeons was explained by measurement noise, whereas the remaining 14% represented true signal. Risk-adjusted complication rates varied from 0% to 55.1% across quartiles before adjusting for reliability. Reliability adjustment greatly diminished this variation, generating a 1.2-fold difference (21.4%-25.6%). A caseload of 168 colectomies across 3 years was required to achieve a reliability of more than 0.7, which is considered a proficient level. Only 1 surgeon surpassed this volume threshold.
Conclusions:
The vast majority of surgeons do not perform enough colectomies to generate a reliable surgeon-specific complication rate. Risk-adjusted complication rates should be viewed with caution when evaluating surgeons with low operative volume, as statistical noise is a large determinant in estimating their surgeon-specific complication rates.
Insights
Most surgeons lack sufficient colectomy volume for reliable complication rate assessment. Statistical noise significantly impacts performance metrics for low-volume surgeons, necessitating cautious interpretation of quality measures.
Area of Science:
- Surgical Quality Measurement
- Health Services Research
- Statistical Modeling in Healthcare
Background:
- Surgeon-specific performance metrics often overlook statistical noise, potentially leading to inaccurate quality assessments.
- Reliable evaluation of surgical quality requires accounting for random variation in outcomes.
Purpose of the Study:
- To assess the reliability of surgeon-specific postoperative complication rates following colectomy.
- To determine the impact of statistical noise on surgeon performance metrics.
Main Methods:
- Analysis of 5033 colectomy patients (2008-2010) from the Michigan Surgical Quality Collaborative Colectomy Project.
- Risk-adjusted complication rates calculated using logistic regression and hierarchical modeling.
- Reliability adjustment applied to surgeon-specific rates to quantify true signal versus noise.
Main Results:
- 86% of variability in complication rates was attributed to measurement noise, with only 14% representing true surgeon performance.
- Reliability adjustment significantly reduced the observed variation in complication rates across surgeons.
- A minimum caseload of 168 colectomies over 3 years was needed for a reliable outcome measure (reliability > 0.7), a threshold surpassed by only one surgeon.
Conclusions:
- The majority of surgeons do not perform enough colectomies to establish a dependable surgeon-specific complication rate.
- Risk-adjusted complication rates for low-volume surgeons should be interpreted with caution due to the substantial influence of statistical noise.
