Related Experiment Videos
Response of general practitioners to computer-generated critiques of hypertension therapy
J van der Lei1, E van der Does, A J Man in 't Veld
1Department of Medical Informatics, Faculty of Medicine and Health Sciences, Erasmus University Rotterdam, The Netherlands.
Insights
Computer systems like HyperCritic can audit hypertension treatment using patient records. General practitioners found computer-generated critiques as useful as human ones, though individual opinions varied significantly.
Area of Science:
- Medical Informatics
- Health Services Research
Background:
- Computer systems can audit clinical practice by analyzing electronic medical records.
- Understanding physician responses to computer-generated critiques is crucial for system development.
Purpose of the Study:
- To compare the perceived relevance of unsolicited critiques from a computer system (HyperCritic) versus human reviewers in hypertension management.
- To assess general practitioners' (GPs) perceptions of computer-generated critiques.
Main Methods:
- Twenty hypertension patient charts (243 visits) were audited by HyperCritic and 8 human reviewers.
- A panel of 14 GPs rated the relevance of critiques on a 5-point scale.
- Critique relevance was categorized as relevant, somewhat relevant, erroneous, or possibly erroneous.
Main Results:
- GPs judged human reviewers' comments as relevant/somewhat relevant in 61-68% of cases (erroneous/possibly erroneous in 15-18%).
- GPs judged HyperCritic's comments as relevant/somewhat relevant in 65% of cases (erroneous/possibly erroneous in 16%).
- Significant inter-rater variability was observed; one GP found 0% of HyperCritic's critiques relevant, while another found 82% relevant.
Conclusions:
- HyperCritic critiques are perceived by GPs as equally beneficial as human critiques.
- Individual GP judgment of critique relevance varies widely.
- Further research is needed to understand reasons for perceived irrelevance and the impact of critiques on physician behavior before widespread adoption.
Abstract:
We recently have shown that a computer system, known as HyperCritic, can successfully audit general practitioners' treatment of hypertension by analyzing computer-based patient records. HyperCritic reviews the electronic medical records and offers unsolicited advice. To determine which unsolicited advice might be perceived as inappropriate, builders of programs such as HyperCritic need insight into providers' responses to computer-generated critique of their patient care. Twenty medical charts, describing in total 243 visits of patients with hypertension, were audited by 8 human reviewers and by the critiquing-system HyperCritic. A panel of 14 general practitioners subsequently judged the relevance of those critiques on a five-point scale ranging from relevant critique to erroneous or harmful critique. The panel judged reviewers' comments to be either relevant or somewhat relevant in 61 to 68% of cases, and either erroneous or possibly erroneous in 15 to 18%; the panel judged HyperCritic's comments to be either relevant or somewhat relevant in 65% of cases, and either erroneous or possibly erroneous in 16%. Comparison of individual members of the panel showed large differences; for example, the portion of HyperCritic's comments judged relevant ranged from 0 to 82%. We conclude that, from the perspective of general practitioners, critiques generated by the critiquing system HyperCritic are perceived equally beneficial as critiques generated by human reviewers. Different general practitioners, however, judge the critiques differently. Before auditing systems based on computer-based patient records that are acceptable to practitioners can be introduced, additional studies are needed to evaluate the reasons a physician may have for judging critiques to be irrelevant, and to evaluate the effect of critiques on physician behavior.