Content validity and inter-rater reliability of an instrument to characterize unintentional medication discrepancies

Medication discrepancies are medication-related problems (MRPs) that frequently occur when patients are transferred between settings of care. Older patients are at high risk for several reasons, including high consumption of medicines, and physical and cognitive deficiencies that can impair the communication process. Most previous studies that have evaluated medication discrepancies used instruments designed for clinical practice, but a well-validated and reliable instrument for clinical research is still lacking.

The aims of this study were to (i) develop an instrument to characterize medication discrepancies that fulfils quality requirements for classification of MRPs related to continuity of care and (ii) assess its content validity and inter-rater reliability.

The instrument was developed based on three main inputs: (i) a literature review to collect information about the quality requirements of instruments to characterize MRPs; (ii) another literature review to identify existing instruments to characterize MRPs and, more specifically, medication discrepancies; and (iii) previous experience from a pilot study on Belgian patients discharged from surgical and medical wards. Content validity was assessed using a modified Delphi technique with 11 healthcare professionals. Content validity indexes were calculated. For inter-rater reliability, three pharmacists (one experienced and two naive) were asked to identify and categorize (type and cause of) unintentional medication discrepancies for 21 patients discharged from hospital into the community. The intra-class correlation coefficient was calculated to compare the number of discrepancies identified, and a paradox-resistant index (AC1) was used to determine the inter-rater reliability for the type and cause of the discrepancy.

The instrument had 54 items classified in three sections (type of discrepancy, cause and intervention), with detailed specifications on how to use it. All evaluations relative to content validity met predefined cut-off values, except for two of them. Intra-class correlation coefficients of ≥0.76 and AC1 coefficients of ≥0.89 were found for the number and the type of discrepancies, respectively. Regarding evaluation of the specific causes of medication discrepancies, final AC1 results of ≥0.86 were obtained, except for three items (which had values between 0.62 and 0.79).

The validity and reliability of the instrument developed to assess unintentional medication discrepancies at patient transition from the hospital to the community setting was found to be satisfactory.