Archival data in program evaluation and policy analysis

Evaluators have typically avoided using existing data sets, choosing instead to collect their own information to maintain control over both the content and quality of the data. As demands on action researchers increase without provision for additional resources, primary data collection may become a luxury and the use of archival data may increase. Though archival data has practical and methodological advantages, there are limitations associated with the utilization of such information. The general problems with secondary data sources include the accuracy, acceptability and accessibility of the information. Following a discussion of these general problems, an example from the Civilian Health and Medical Program of the Uniformed Services (CHAMPUS) reimbursement data set is presented to illustrate specific difficulties with using archival data for evaluation research. Strategies are then presented for minimizing these difficulties, including determining the feasibility of utilization of such data sources, methods for assessing their accuracy and factors to consider in the data acquisition process.