Policy and program analysis using administrative databases

Area of Science:

• Health Services Research
• Health Policy Analysis
• Medical Care Evaluation

Background:

• Administrative policies significantly influence medical care utilization.
• Randomized controlled trials for policy evaluation are rare due to political and logistical barriers.
• Retrospective evaluations using administrative databases are common but face data quality and methodological challenges.

Purpose of the Study:

• To highlight the limitations of retrospective evaluations of administrative healthcare policies.
• To propose strategies for improving the validity of such evaluations.
• To emphasize the need for more frequent prospective evaluations.

Main Methods:

• Review and illustration of problems using published evaluations of administrative policies.
• Discussion of strategies to minimize evaluation impact issues.
• Focus on data quality assessment, interrupted time-series analysis, and use of medical encounter data.

Main Results:

• Retrospective evaluations often suffer from poor data quality, lack of concurrent controls, and incomplete case-mix data.
• Proposed strategies include rigorous data quality checks, advanced analytical methods, and focused outcome selection.
• Even with mitigation strategies, comprehensive evaluation may remain difficult.

Conclusions:

• Evaluating the impact of administrative policies on medical care is complex and often hindered by data limitations.
• While improved methods can enhance retrospective analyses, prospective evaluations are essential for robust evidence.
• Adopting a standard of rigorous evaluation, similar to therapeutic interventions, is necessary for healthcare policies.