Transparency and Authority Concerns with Using AI to Make Ethical Recommendations in Clinical Settings

In response to recent proposals to utilize artificial intelligence (AI) to automate ethics consultations in healthcare, we raise two main problems for the prospect of having healthcare professionals rely on AI-driven programs to provide ethical guidance in clinical matters. The first cause for concern is that, because these programs would effectively function like black boxes, this approach seems to preclude the kind of transparency that would allow clinical staff to explain and justify treatment decisions to patients, fellow caregivers, and those tasked with providing oversight. The other main problem is that the kind of authority that would need to be given to the guidance issuing from these programs in order to do the work set out for them would mean that clinical staff would not be empowered to provide meaningful safeguards against it in those cases when its recommendations are morally problematic.