Benchmarking clinical practice in surgery: looking beyond traditional mortality rates

Area of Science:

• Medical Statistics
• Surgical Quality Improvement
• Health Services Research

Background:

• Traditional risk-adjusted mortality rates (RAMRs) have limitations in fully assessing surgical practice performance.
• Evaluating surgical outcomes requires nuanced metrics that account for patient risk stratification.
• Existing performance measures may not adequately differentiate between high-risk and low-risk patient outcomes.

Purpose of the Study:

• To introduce and validate two novel performance measures for surgical practice: reliability and living score.
• To demonstrate how these measures can provide a more comprehensive assessment of surgical performance compared to RAMRs.
• To explore the application of these measures in improving surgical risk management and patient care.

Main Methods:

• Development of two new performance measures: reliability (Area Under the ROC Curve) and living score (risk-adjusted survival).
• Conceptual validation through Monte Carlo simulations of surgical practice under various conditions.
• Managerial validation via analysis of a real-world Portuguese cardiothoracic department's surgical data.

Main Results:

• The proposed reliability and living score measures offer a more detailed valuation of surgical outcomes than RAMRs by differentiating deaths based on risk levels.
• Reliability favors surgeons with high-risk patient mortality and penalizes those with low-risk patient mortality.
• The living score is positively influenced by the highest risk level at which a surgeon achieves patient survival, indicating effective risk management.
• Application in a cardiothoracic department suggested performance improvements through enhanced risk management policies and optimized surgery assignment based on surgeon reliability and living score.

Conclusions:

• The joint application of reliability and living score provides a superior assessment of surgical practice compared to RAMRs.
• These measures rely solely on mortality and risk score data, making them practical for widespread implementation.
• Implementing performance-based surgical assignments using reliability and living scores can enhance departmental efficiency and patient outcomes.