Comprehensive Evaluation of Quality Indicators: Analyzing the Dutch Breast Cancer Audit

Area of Science:

• Oncology
• Healthcare Quality Improvement
• Health Services Research

Background:

• Quality indicators (QIs) are crucial for hospital performance benchmarking and care improvement.
• Current QIs often rely on expert opinion rather than data-driven validation.
• A systematic evaluation framework is needed to assess QI quality.

Purpose of the Study:

• To evaluate existing breast cancer quality indicators using a data-driven framework.
• To assess QIs based on feasibility, discriminative ability, validity, and reliability.
• To identify areas for improvement in QI development and application.

Main Methods:

• Utilized data from the Dutch breast cancer registry (NABON Breast Cancer Audit, NBCA) for surgically treated patients (2021-2023).
• Evaluated 18 QIs based on predefined criteria: feasibility (>90% data completeness), discriminative ability (IQR >10%), validity (case-mix adjustment impact, pseudo-R² <0.10), and reliability (rankability >75%).

Main Results:

• Fifteen of 18 QIs were feasible (80.2%-100% data availability).
• Eight QIs demonstrated good discriminative ability (IQR >10%).
• Eleven QIs showed low case-mix impact (validity), but none met the high reliability threshold (rankability 0-69%).

Conclusions:

• While Dutch breast cancer QIs largely meet feasibility and validity criteria, reliability is a significant concern for performance benchmarking and public reporting.
• The study underscores the importance of a comprehensive evaluation framework (feasibility, discriminative ability, validity, reliability) for QI development and assessment.
• Specific attention is needed to enhance the reliability of QIs to ensure accurate quality assessment.