Assessing the quality of project data by means of a data sufficiency categorization scheme

Improving the quality of gathered or abstracted data is often an important part of a quality improvement project's early stages. This is especially so if indicators will be compared across providers or if various data elements will be used for severity adjustment. One common data problem is missing data. This paper describes a flowchart-based approach for assessing the magnitude of missing data problems. The approach is demonstrated by means of two indicators currently used in multihospital cooperative improvement projects. The approach results in assigning each observation (e.g., patient record) in a sample into one of five data categories. These categories follow standard definitions advanced by the Joint Commission on Accreditation of Healthcare Organizations. Two of the categories tally observations with missing data problems. Assessment of missing data can be viewed as one component of reliability assessment, and its relationship to other forms of reliability assessment is discussed, with emphasis on the relationship to interrater agreement.