Validity and reliability issues in alternative patient classification systems

The validity and reliability issues involved in using alternative patient classification systems were reviewed. Disease Staging and Patient Management Categories (PMCs) were applied separately and in conjunction with DRGs to three populations of patients drawn from a nine-hospital community data base. Data were examined with analyses that were as consistent as possible with hospital-based reviews of resource utilization. Questions focused on content and context validity (partially assessable by homogeneity), general and statistical reliability (measured by variance reduction), gaming, and cost. Ordinal stratifications were inconsistently produced, and improvement to DRGs' homogeneity was generally negligible. When used alone, staging produced only half the variance reduction of DRGs. PMCs, when used alone, appeared to produce sizeable variance reductions that may have been due to the large number of one- and two-case categories produced. Staging had category overlap, was expensive, and was unidimensional and subject to manipulation. PMCs had potentially serious logic problems, and both were inadequately documented. Neither system was considered appropriate for all needs, but each might work adequately under well-defined and limited conditions.