Reliability study of the European appropriateness evaluation protocol

Area of Science:

• Health Services Research
• Medical Informatics
• Quality Improvement

Background:

• Healthcare utilization review is crucial for resource management and quality assurance.
• The US Appropriateness Evaluation Protocol (AEP) is a widely used tool for assessing healthcare utilization.
• Adapting international tools requires rigorous validation for regional applicability.

Purpose of the Study:

• To adapt the US Appropriateness Evaluation Protocol (AEP) for European healthcare settings, creating the EU-AEP.
• To assess the inter-rater reliability of the European version of the AEP (EU-AEP).
• To evaluate the EU-AEP's suitability for harmonizing and coordinating utilization review research in Europe.

Main Methods:

• Nineteen medical records were reviewed by six physician reviewers from Austria, France, Italy, Spain, Switzerland, and the UK.
• Reviewers assessed the appropriateness of 19 admissions and 31 hospitalization days using the EU-AEP instrument.
• Inter-rater reliability was evaluated using the kappa statistic for admission and day-of-care appropriateness.

Main Results:

• The overall kappa statistic for admission appropriateness was 0.64 (range: 0.46-0.86).
• The overall kappa statistic for day-of-care appropriateness was 0.59 (range: 0.25-0.95).
• These kappa values indicate substantial agreement among reviewers.

Conclusions:

• The EU-AEP demonstrates substantial reliability in the European context, supporting its use in utilization review.
• The instrument is suitable for comparative studies involving multiple European countries.
• Preliminary findings support the EU-AEP as a reliable tool for harmonizing European healthcare research despite study limitations.