Implementing artificial intelligence in chest diagnostics for lung disease: A mixed-methods evaluation

Artificial intelligence tools simulate aspects of human intelligence. Policy and research highlight artificial intelligence's potential to support delivery of radiology pathways. In 2023, National Health Service (NHS) England invested £21M to deploy artificial intelligence diagnostic tools for chest X-ray and chest computed tomography in 66 NHS Trusts. Little is known about how artificial intelligence tools are implemented in practice, staff experience of these tools, and their effectiveness and cost.

Evaluate evidence on artificial intelligence tools within radiology internationally. Evaluate implementation of artificial intelligence for chest diagnostics in England. Investigate how effectiveness and cost-effectiveness of artificial intelligence for chest diagnostics can be measured.

Ten-month mixed-methods study (rapid scoping review and empirical study comprising staff interviews, observations and documentary analysis). Findings were analysed using rapid assessment procedures, drawing on qualitative, quantitative and health economic approaches. Our evaluation was also designed to inform phase 2 of our study.

The review included 114 articles on artificial intelligence use in radiology, internationally. Empirical work included 51 staff interviews, 57 observations and 166 documents from 10/11 of the networks and 6/66 trusts implementing artificial intelligence tools. Our review found evidence gaps, including real-world implementation of artificial intelligence tools; patient and carer experiences; impact on inequalities, sustainability and wider systems; and cost-effectiveness. Artificial intelligence for chest diagnostics was implemented in various ways, with different aims, pathways and approaches. Implementation takes time due to multiple activities related to planning, procurement, preparation for deployment, monitoring and evaluation. These tasks - technical and social - required time and resource, including a wide range of stakeholders and expertise. As of November 2024, 24/66 trusts had implemented artificial intelligence tools in practice. Implementation barriers included time, resources, challenges navigating processes and adapting these to local contexts. Facilitators included stakeholder engagement and support. Network and trust ability to evaluate service impact was influenced by factors such as data availability, data linkage, resources and capacity. Factors varied across implementation stages. Our findings indicated multiple data sources that may support measurement of effectiveness and cost-effectiveness within the English National Health Service. However, limitations to data availability need to be addressed.

Our rapid timeline meant we could not interview patients, carers, and several staff groups at trust and national levels. Delayed deployment meant we could not study implementation in practice or staff experiences.

Should address real-world implementation, adaptation, and sustainability of artificial intelligence tools; impact of artificial intelligence on care, outcomes, and cost-effectiveness; and staff, patient, and carer experiences of artificial intelligence in practice. We will study these issues in phase 2 of our evaluation.

Artificial intelligence tools may support effective, efficient chest diagnostics services. However, several factors should be considered when implementing and monitoring artificial intelligence tools. Implementation and monitoring may be improved through allowing sufficient time for procurement and preparation for deployment or extending capacity to speed completion of these tasks, early and ongoing stakeholder engagement, sufficient resourcing, dedicated expertise and clinical champions, simplifying governance processes, and improving data capacity. Parallels with learning from implementing other innovations suggests that artificial intelligence tools may not offer straightforward solutions anticipated by services and policy-makers.

This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme as award number NIHR167339.