The evaluation of the National Long Term Care Demonstration. 2. Estimation methodology

Channeling effects were estimated by comparing the post-application experience of the treatment and control groups using multiple regression. A variety of potential threats to the validity of the results, including sample composition issues, data issues, and estimation issues, were identified and assessed. Of all the potential problems examined, the only one determined to be likely to cause widespread distortion of program impact estimates was noncomparability of the baseline data. To avoid this distortion, baseline variables judged to be noncomparably measured were excluded from use as control variables in the regression equation. (Where they existed, screen counterparts to these noncomparable baseline variables were used as substitutes.) All of the other potential problems with the sample, data, or regression estimation approach were found to have little or no actual effect on impact estimates or their interpretation. Broad implementation of special procedures, therefore, was not necessary. The study did find that, because of the frequent use of proxy respondents, the estimated effects of channeling on clients' well-being actually may reflect impacts on the well-being of the informal caregiver rather than the client. This and other isolated cases in which there was some evidence of a potential problem for specific outcome variables were identified and examined in detail in technical reports dealing with those outcomes. Where appropriate, alternative estimates were presented.