Establishing Trust in Pharmaceutical Data with an Independent Verification and Validation Methodology

Area of Science:

• Health Informatics
• Data Science
• Quality Management

Background:

• Electronic Health Records (EHRs) facilitate data exchange but require robust validation during migration.
• Ensuring data quality and safety is critical in healthcare to maintain patient safety and treatment efficacy.
• Existing data validation methods may not adequately address the complexities of health data, particularly unstructured text.

Purpose of the Study:

• To demonstrate a general-purpose approach for data and knowledge validation.
• To establish reproducible metrics for assessing data and knowledge quality and safety.
• To apply statistical process control methods to pharmacy prescription data during EHR migration.

Main Methods:

• Researched statistical process control (SPC) methods from high-safety industries.
• Applied SPC methods to pharmacy prescription data undergoing EHR migration.
• Two terminologists independently categorized natural language medication instructions for standardization.

Main Results:

• A weighted average of 43% for matched medication instructions by reviewers.
• High inter-rater reliability for short (K=0.82) and long (K=0.85) instructions.
• Moderate inter-rater reliability for medium instructions (K=0.61), indicating areas for refinement.

Conclusions:

• The study demonstrates a novel approach to health data validation using SPC.
• Refining category definitions is necessary to improve agreement in instruction categorization.
• Incorporating statistical tests for inter-rater reliability is crucial for developing health data quality benchmarks.