Creation of a gold standard for validation of a computer-interpretable depression screening guideline

Clinical practice guidelines (CPGs) are common tools for clinicians in daily practice. In order to use CPGs effectively at the point of care, representing CPGs into computer-interpretable format is essential. Since computer-interpretable guidelines (CIGs) have been reported to increase clinicians' usage of guidelines and improve patient's outcomes, it is critical to validate health care knowledge translated from CPGs into CIGs. A comprehensive method of developing and evaluating a gold standard for a depression screening CIG was performed and analyzed in this study. GLIF encoding was conducted for a depression screening CPG and 21 clinical scenarios were created based on the encoded depression screening CIG. Two nurse practitioners were recruited to generate initial management for each scenario. Proficiency and efficiency scores were calculated for each scenario. In 13 of 21 scenarios, both experts had proficiency scores of 100%. Proficiency scores were lowest for scenarios that included bipolar disease. Implications of our findings for development of a gold standard are discussed.