Quality control of resident operative experience: compliance with RRC criteria

Process Capability Analysis (PCA) is a quality control tool that can be applied to how resident cases are allocated. PCA measures how close an output is to its target (Cp) and its location (K) in relation to Cp. For resident cases, the statistics of Cp and K measure variability in case numbers, which is useful in planning how a program meets RRC criteria in operative experience on specialty services.

A review of 6 years of vascular surgery cases (1994-1999) using PG 5 RRC case logs and the departmental quality database as sources was done. PCA was applied to resident case numbers. RRC 1997-1998 National Program Data were used to define quality control limits. The 30th, 50th, and 70th percentiles in vascular procedures served as lower, nominal, and upper control limits, respectively.

Cases were grouped into aortic (AO), cerebrovascular (CER), peripheral (PER), and visceral (VSC), to meet RRC definition for major reconstructions. PCA analysis of PG5 RRC submitted data, total cases reported by the graduating residents (n = 20) in their submitted RRC Case logs, and department QA log numbers over 6 years of the period revealed the following mean numbers for AO = 13.7 +/- 4.2 (Cp = 0.279, K = +70.5%), CER = 15.6 +/- 6.4 (Cp = 0.365, K = +78.6%), PER = 23.1 +/- 10.5 (Cp = 0.237, K = +80.1%), VSC = 2.5 +/- 1.5 (Cp = 0.189, K = +175.0%), and total = 55.1 +/- 20.2 (Cp = 0.305, K = +70.5%).

Processes that are in statistical control should have a Cp > 1.0, and a small K: i.e., centered tightly and near the desired nominal limit. By PCA analysis, the vascular surgery service product of PG5 case numbers is not in statistical control, but instead exceeds the defined control limits (RRC 50th percentile levels). The low Cp values demonstrate that specific case numbers may vary from resident to resident. The high positive K percentages imply that more residents have case numbers that meet or exceed RRC 50th percentiles. Sources of potential error and variability may be partially explained by the data of total case volume and resident underreporting. More accurate RRC reporting by residents could account for up to 20% more cases, potentially offsetting individual case allocation differences within a particular resident year.