Related Experiment Videos
Consistency of evaluators assessing inhaler technique
M B O'Connell1, J M Hewitt, T E Lackner
1Department of Pharmacy Practice, University of Minnesota Pharmacy Department, Hennepin County Medical Center, Minneapolis.
Annals of Allergy
|December 1, 1991
Summary
Achieving high interrater reliability in assessing inhaler technique requires specific training and checklist refinement. This study improved agreement from 81% to 98% through iterative revisions and evaluator training.
Area of Science:
- Respiratory medicine
- Medical education
- Clinical research methodology
Background:
- Correct inhaler technique is crucial for medication efficacy.
- Inconsistent assessment of inhaler technique can lead to patient confusion and flawed research.
- Standardized evaluation methods are needed for reliable assessment.
Purpose of the Study:
- To determine interrater consistency in assessing inhaler technique (recitation and demonstration).
- To improve assessment consistency to at least 95% agreement.
- To establish reliable methods for evaluating inhaler technique in research and education.
Main Methods:
- Developed a checklist for correct inhaler technique based on literature and package inserts.
- Conducted six trials involving 56 healthcare professionals across five sites.
- Utilized three, then four evaluators in teams, refining the checklist and training to enhance agreement.
Main Results:
- Overall percent agreement increased from 81% to 98% after checklist revisions and evaluator training.
- Verbal component agreement improved significantly due to checklist specificity.
- Demonstration component agreement remained lower due to inherent subjectivity.
Conclusions:
- Iterative checklist refinement and evaluator training are essential for achieving high interrater consistency.
- Standardized assessment protocols are necessary for accurate research reporting and effective patient education.
- While challenging, improving consistency in demonstration assessment is vital.