Accuracy of administrative data versus clinical data to evaluate carotid endarterectomy and carotid stenting
Rodney P Bensley1, Shunsuke Yoshida, Ruby C Lo
1Department of Surgery, Division of Vascular and Endovascular Surgery, Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, MA 02215, USA.
Insights
Administrative data are unreliable for evaluating carotid endarterectomy (CEA) and carotid artery stenting (CAS) outcomes. The National Surgical Quality Improvement Program (NSQIP) data better identify strokes and symptom status but not high-risk patients.
Area of Science:
- Vascular Surgery
- Health Services Research
- Data Quality Assessment
Background:
- Administrative data are commonly used for comparing carotid endarterectomy (CEA) and carotid artery stenting (CAS).
- Limitations exist in administrative data for accurately defining patient symptom status, high-risk criteria (e.g., Centers for Medicare and Medicaid Services), and perioperative complications.
- Direct comparison of administrative data and NSQIP data against physician chart review is necessary to assess their reliability.
Purpose of the Study:
- To directly compare the accuracy of administrative data versus physician chart review for assessing symptom status, high-risk status, and perioperative stroke in CEA and CAS.
- To compare the accuracy of National Surgical Quality Improvement Program (NSQIP) data versus physician chart review for the same parameters in CEA.
Main Methods:
- Outcomes analysis of CEA and CAS procedures from 2005-2011 using International Classification of Diseases, Ninth Revision codes from hospital discharge records.
- Collection of CEA patient data submitted to NSQIP during the same period.
- Independent physician chart review by a study author to determine symptom status, high-risk status, and perioperative strokes for comparison with administrative and NSQIP data.
Main Results:
- Administrative data underestimated symptomatic patients (17.0% vs 34.0%) and high-risk patients (physiologic 9.3% vs 23.0%; anatomic 0% vs 15.2%) compared to chart review.
- NSQIP data identified more symptomatic patients (44.1% vs 30.3%) but fewer high-risk patients (physiologic 13.0% vs 18.6%; anatomic 0% vs 6.6%) than chart review.
- Both administrative and NSQIP data showed similar proportions of perioperative strokes, but administrative data had significant false positives and negatives, while NSQIP had one false negative.
Conclusions:
- Administrative data are unreliable for assessing symptom status, high-risk status, and perioperative stroke in CEA and CAS and should not be used for such analyses.
- NSQIP data accurately identify perioperative strokes and moderately well identify symptom status.
- NSQIP data are insufficient for adequately identifying high-risk patients undergoing CEA.
Objective:
Administrative data have been used to compare carotid endarterectomy (CEA) and carotid artery stenting (CAS). However, there are limitations in defining symptom status, Centers for Medicare and Medicaid Services high-risk status, as well as complications. Therefore, we did a direct comparison between administrative data and physician chart review as well as between data collected for the National Surgical Quality Improvement Program (NSQIP) and physician chart review for CEA and CAS.
Methods:
We performed an outcomes analysis on all CEA and CAS procedures from 2005 to 2011. We obtained International Classification of Diseases, Ninth Revision diagnosis codes from hospital discharge records regarding symptom status, high-risk status, and perioperative stroke. We also obtained data on all CEA patients submitted to NSQIP over the same time period. One of the study authors (R.B.) then performed a chart review of the same patients to determine symptom status, high-risk status, and perioperative strokes and the results were compared.
Results:
We identified 1342 patients who underwent CEA or CAS between 2005 and 2011 and 392 patients who underwent CEA that were submitted to NSQIP. Administrative data identified fewer symptomatic patients (17.0% vs 34.0%), physiologic high-risk patients (9.3% vs 23.0%), and anatomic high-risk patients (0% vs 15.2%). Although administrative data identified a similar proportion of perioperative strokes (1.9% vs 2.0%), this was due to the fact that these data identified eight false positive and nine false negative perioperative strokes. NSQIP data identified more symptomatic patients compared with chart review (44.1% vs 30.3%), fewer physiologic high-risk patients (13.0% vs 18.6%), fewer anatomic high-risk patients (0% vs 6.6%), and a similar proportion of perioperative strokes (1.5% vs 1.8%, only one false negative stroke and no false positives).
Conclusions:
Administrative data are unreliable for determining symptom status, high-risk status, and perioperative stroke and should not be used to analyze CEA and CAS. NSQIP data do not adequately identify high-risk patients, but do accurately identify perioperative strokes and to a lesser degree, symptom status.
Related Concept Videos
01:27Coronary Artery Disease V: Interprofessional Care
01:27Atherosclerosis II: Clinical Manifestations and Diagnostic Tests
01:25Imaging Studies for Cardiovascular System VI: Calcium -Scoring CT