Developing Criteria and Associated Instructions for Consistent and Useful Quality Improvement Study Data Extraction...

Area of Science:

• Health Services Research
• Evidence Synthesis
• Quality Improvement Science

Background:

• Learning health systems (LHS) require reliable quality improvement (QI) studies.
• The Agency for Healthcare Research and Quality (AHRQ) could support LHS by collating QI studies.
• Current data extraction consistency for QI studies lacks reliable assessment.

Purpose of the Study:

• To evaluate the consistency of data extraction for QI studies.
• To assess the impact of explicit instructions on data extraction consistency.
• To gauge the perceived value of AHRQ support for LHS.

Main Methods:

• Two independent reviewers extracted data from QI studies at baseline and after two revisions of explicit instructions.
• Six investigators assessed data extraction similarity (0-10 scale).
• Two LHS participants rated the value of AHRQ's potential support.

Main Results:

• Data extraction consistency improved significantly from baseline (1.17) to Revision 1 (6.07) and Final Revision (6.81).
• The final revision showed no significant improvement over the first revision (P=0.14).
• LHS participants rated AHRQ's collation of QI study data as highly valuable (9/10 and 6/10).

Conclusions:

• Developing explicit instructions enhances data extraction consistency for QI studies.
• Improved consistency is crucial for LHS to effectively vet QI studies.
• AHRQ support in collating QI study data is highly valued by LHS.