Validation of a Crowdsourcing Methodology for Developing a Knowledge Base of Related Problem-Medication Pairs

Area of Science:

• Health Informatics
• Clinical Decision Support
• Knowledge Engineering

Background:

• Clinical knowledge bases of problem-medication pairs are crucial for patient safety informatics solutions.
• Developing these comprehensive knowledge bases presents significant challenges in healthcare.

Purpose of the Study:

• To validate a crowdsourcing methodology for creating a problem-medication knowledge base.
• To assess this approach within a large healthcare system utilizing a commercial electronic health record.

Main Methods:

• Extracted problem-medication pairs from electronic health records over six months.
• Applied link frequency and ratio analysis, with clinician review to establish a threshold for pair appropriateness.
• Compared the generated knowledge base against a gold standard of 50 medication-indication pairs.

Main Results:

• The crowdsourced knowledge base comprised 26,912 problem-medication pairs.
• Achieved a recall of 62.3% and a precision of 87.5%.
• Outperformed a previous pilot knowledge base (11,167 pairs) which had 46.9% recall and 83.3% precision.

Conclusions:

• Validated crowdsourcing for generating problem-medication knowledge bases in a large healthcare setting.
• The approach demonstrates potential generalizability across diverse healthcare environments and clinical systems.
• Further research is needed to evaluate knowledge quality, compare with alternative methods, and assess impact on patient outcomes.