Related Experiment Video
Updated: Oct 29, 2025
Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
Systematic Undercoding of Diagnostic Procedures in National Inpatient Sample (NIS): A Threat to Validity Due to
Oluwafemi P Owodunni1, Brandyn D Lau, Katherine L Florecki
1Division of Acute Care Surgery, Department of Surgery (Drs Owodunni, Florecki, Webster, and Haut and Ms Holzmueller), Department of Surgery (Mss Shaffer and Hobson), Department of Anesthesiology and Critical Care Medicine (Dr Haut), and Department of Emergency Medicine (Dr Haut), The Johns Hopkins Surgery Center for Outcomes Research, Baltimore, Maryland (Mr Canner); Division of Hematology, Department of Medicine (Dr Streiff), Russell H. Morgan Department of Radiology and Radiological Science (Mr Lau), and Division of Health Sciences Informatics (Mr Lau), The Johns Hopkins University School of Medicine, Baltimore, Maryland; Departments of Nursing (Mss Shaffer and Hobson) and Pharmacy (Dr Kraus), The Johns Hopkins Hospital, Baltimore, Maryland; The Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, Baltimore, Maryland (Drs Haut and Streiff, Mss Hobson and Holzmueller, and Mr Lau); and Department of Health Policy and Management, The Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland (Dr Haut and Mr Lau).
Background And Objectives:
Health services research often relies on readily available data, originally collected for administrative purposes and used for public reporting and pay-for-performance initiatives. We examined the prevalence of underreporting of diagnostic procedures for acute myocardial infarction (AMI), deep venous thrombosis (DVT), and pulmonary embolism (PE), used for public reporting and pay-for-performance initiatives.
Method:
We retrospectively identified procedures for AMI, DVT, and PE in the National Inpatient Sample (NIS) database between 2012 and 2016. From January 1, 2012, through September 30, 2015, the NIS used the International Classification of Diseases, Ninth Revision (ICD-9) coding scheme. From October 1, 2015, through December 31, 2016, the NIS used the International Classification of Diseases, Tenth Revision (ICD-10) coding scheme. We grouped the data by ICD code definitions (ICD-9 or ICD-10) to reflect these code changes and to prevent any confounding or misclassification. In addition, we used survey weighting to examine the utilization of venous duplex ultrasound scan for DVT, electrocardiogram (ECG) for AMI, and chest computed tomography (CT) scan, pulmonary angiography, echocardiography, and nuclear medicine ventilation/perfusion () scan for PE.
Results:
In the ICD-9 period, by primary diagnosis, only 0.26% (n = 5930) of patients with reported AMI had an ECG. Just 2.13% (n = 7455) of patients with reported DVT had a peripheral vascular ultrasound scan. For patients with PE diagnosis, 1.92% (n = 12 885) had pulmonary angiography, 3.92% (n = 26 325) had CT scan, 5.31% (n = 35 645) had cardiac ultrasound scan, and 0.45% (n = 3025) had scan. In the ICD-10 period, by primary diagnosis, 0.04% (n = 345) of reported AMI events had an ECG and 0.91% (n = 920) of DVT events had a peripheral vascular ultrasound scan. For patients with PE diagnosis, 2.08% (n = 4805) had pulmonary angiography, 0.63% (n = 1460) had CT scan, 1.68% (n = 3890) had cardiac ultrasound scan, and 0.06% (n = 140) had scan. Small proportions of diagnostic procedures were observed for any diagnoses of AMI, DVT, or PE.
Conclusions:
Our findings question the validity of using NIS and other administrative databases for health services and outcomes research that rely on certain diagnostic procedures. Unfortunately, the NIS does not provide granular data that can control for differences in diagnostic procedure use, which can lead to surveillance bias. Researchers and policy makers must understand and acknowledge the limitations inherent in these databases, when used for pay-for-performance initiatives and hospital benchmarking.
Insights
Administrative data often underreports diagnostic procedures for acute myocardial infarction (AMI), deep venous thrombosis (DVT), and pulmonary embolism (PE). This limits the reliability of health services research and pay-for-performance initiatives using these datasets.
Area of Science:
- Health Services Research
- Medical Informatics
- Cardiovascular Research
Background:
- Health services research frequently utilizes administrative data for public reporting and pay-for-performance initiatives.
- The accuracy of these databases for specific diagnostic procedures is often unexamined.
- Underreporting of key diagnostic procedures can impact the validity of research findings.
Purpose of the Study:
- To examine the prevalence of underreported diagnostic procedures for acute myocardial infarction (AMI), deep venous thrombosis (DVT), and pulmonary embolism (PE).
- To assess the impact of coding system changes (ICD-9 to ICD-10) on procedure reporting.
- To evaluate the suitability of administrative data for health services research and performance measurement.
Main Methods:
- Retrospective analysis of the National Inpatient Sample (NIS) database from 2012-2016.
- Data grouped by International Classification of Diseases (ICD-9 and ICD-10) coding periods to manage code changes.
- Utilization of survey weighting to examine specific diagnostic procedures: ECG for AMI, venous duplex ultrasound for DVT, and CT scan, pulmonary angiography, echocardiography, and V/Q scan for PE.
Main Results:
- Very low proportions of reported AMI, DVT, and PE cases had corresponding diagnostic procedures documented in the NIS database.
- Underreporting persisted across both ICD-9 and ICD-10 coding periods.
- Specific findings include: ECG for AMI (0.26% ICD-9, 0.04% ICD-10), peripheral vascular ultrasound for DVT (2.13% ICD-9, 0.91% ICD-10), and various PE diagnostic procedures ranging from 0.06% to 5.31%.
Conclusions:
- Findings question the validity of using administrative databases like NIS for health services and outcomes research relying on diagnostic procedure data.
- The lack of granular data in NIS hinders control for variations in diagnostic procedure use, potentially causing surveillance bias.
- Researchers and policymakers must acknowledge these limitations when using administrative data for pay-for-performance initiatives and hospital benchmarking.
More Related Videos
10:38Observational Study Protocol for Repeated Clinical Examination and Critical Care Ultrasonography Within the Simple Intensive Care Studies
Published on: January 16, 2019
07:31Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
Related Concept Videos
01:10Documentation of Nursing Diagnosis
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
02:07Blind Procedures
01:09Censoring Survival Data
01:29Bias in Epidemiological Studies
01:26Principles of Disease Surveillance
01:23Methods of Documentation V: CBE
In CBE, healthcare professionals establish predefined standards of practice that define what constitutes...