Artificial intelligence software to help detect fractures on X-rays in urgent care: An Early Value Assessment

Artificial intelligence algorithms have been developed to support clinicians in diagnosing fractures, with the intention to improve the diagnostic accuracy of clinicians reviewing X-rays. The purpose of this rapid early value assessment was to identify the existing evidence base for the technology and to assess whether there was a prima facie case for the technology to represent positive outcomes for patients and a value-for-money investment for people in the National Health Service.

This early value assessment assessed the potential value of the use of artificial intelligence to aid clinician diagnosis of fractures in emergency care settings as compared to clinician-diagnosis alone. A rapid evidence review was conducted followed by 'light touch' early economic modelling to explore whether a plausible case could be made for cost-effectiveness at the prices charged by the companies. Evidence searches were conducted in June and July 2024 to identify clinical, diagnostic and service outcomes associated with the technology. A simple decision model incorporating prevalence, sensitivity, specificity and cost per scan for each of the technologies was developed to evaluate plausible cost-effectiveness for detecting ankle and foot, wrist and hand, and hip fractures, selected based on the availability of evidence and their downstream costs and consequences.

Sixteen studies identified evaluated the diagnostic accuracy of the technology. None of the included studies were conducted in the United Kingdom and all were associated with limitations. While the studies were not considered to be able to provide reliable estimates of diagnostic accuracy, there was a trend for the technology to improve sensitivity for detecting fractures. The technology had no discernible impact on the rate of false-positive diagnoses. Overall, most of the evaluated technologies were associated with a positive incremental net health benefit at willingness-to-pay thresholds of £20,000 and £30,000 per quality-adjusted life-year gained. Due to data limitations, it was not possible to compare technologies against each other. The results were mostly robust to scenario analyses.

The evidence base for the technology is currently limited to studies evaluating diagnostic accuracy and it is unclear whether increases in fracture detection would translate into meaningful benefits for patients and services. While there are some fractures that, if missed, can result in significant harm to patients, it is plausible that the technology would improve diagnosis of more subtle fractures that may not require a change in management. Use of the technology would not eradicate the risk of missed fractures, meaning that health services would need to continue to take precautions to avoid the risk of a missed fracture in clinical practice. A simple decision tree analysis suggested that the technology was plausibly cost-effective at conventional National Institute for Health and Care Excellence thresholds.

There are significant limitations in the available evidence leading to uncertainties about the diagnostic accuracy of the technology within NHS settings. Due to the pragmatic nature of the early value assessment and the available evidence base, the economic analysis included many gross assumptions and was unable to produce a definitive estimate of cost-effectiveness.

The appraisal resulted in a number of research recommendations for evaluating the technology further. More detailed modelling in a full formal diagnostic assessment review is required to consider the longer-term costs and consequences of false negatives and positives, and how they are likely to impact the estimates of cost-effectiveness.

This study is registered as PROSPERO CRD42024574393.

This award was funded by the National Institute for Health and Care Research (NIHR) Evidence Synthesis programme (NIHR award ref: NIHR136024) and is published in full in Health Technology Assessment; Vol. 30, No. 33. See the NIHR Funding and Awards website for further award information.