Clouds on the horizon: clinical decision support systems, the control problem, and physician-patient dialogue

Artificial intelligence-based clinical decision support systems have a potential to improve clinical practice, but they may have a negative impact on the physician-patient dialogue, because of the control problem. Physician-patient dialogue depends on human qualities such as compassion, trust, and empathy, which are shared by both parties. These qualities are necessary for the parties to reach a shared understanding -the merging of horizons- about clinical decisions. The patient attends the clinical encounter not only with a malfunctioning body, but also with an 'unhomelike' experience of illness that is related to a world of values and meanings, a life-world. Making wise individual decisions in accordance with the patient's life-world requires not only scientific analysis of causal relationships, but also listening with empathy to the patient's concerns. For a decision to be made, clinical information should be interpreted considering the patient's life-world. This side of clinical practice is not a job for computers, and they cannot be final decision-makers. On the other hand, in the control problem users blindly accept system output because of over-reliance, rather than evaluating it with their own judgement. This means over-reliant parties leave their place in the dialogue to the system. In this case, the dialogue may be disrupted and mutual trust may be lost. Therefore, it is necessary to design decision support systems to avoid the control problem and to limit their use when this is not possible, in order to protect the physician-patient dialogue.