Impact of a GRADE-based medical question answering system on physician behaviour: a randomised controlled trial

Physicians are frequently faced with questions related to their patients’ care that they cannot answer. A vast number of randomised trials have tested a wide variety of behaviour-changing strategies designed to improve practitioners’ evidence utilisation, but systematic reviews have concluded that the effects are generally small and inconsistent. We conducted a randomised controlled trial to determine whether a question identification and solving system, using structured evidence summaries with recommendations, would change physician’s behavior related to the care of their hospitalised patients. The trial was conducted at the secondary level, internal medicine ward. Relevant clinical questions were the units of randomisation; 14 clinicians participated in the study. The question identification and answering system was carried out using evidence summaries with recommendations based on the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) approach stressing influence on clinician behaviour (decision/recommendation concordance). During 131 morning reports, 553 questions were identified (4.2 questions per meeting). 398 were excluded because they were not about diagnostic or therapeutic interventions or because their answers could not have impact on clinician behaviour, and 31 were excluded because of lack of time to answer them, leaving 124 included questions. The proportion of clinical decisions concordant with the proposed recommendations was 79%in the intervention arm and 44% in the control arm: relative risk 1.8 (95% CI 1.3 to 2.4), number of evidence summaries needed to change a care decision for one question raised was 3 (95% CI 2 to 6). A question identification and answering system was feasible, effectively performed and significantly influenced clinician behaviour related to the care of hospitalised patients, which suggests that interventions facilitating accessibility and interpretability of the best available evidence at the point of care have the potential to significantly impact on the quality of healthcare.