Enhancing clinicians' trust in large language models via transparent source attribution: A randomized controlled...

Large language models (LLMs) are utilized to answer queries in urology and oncology, yet the performance is limited due to outdated data and missing source transparency, which undermines clinical reliability and therefore adoption.

We developed UroBot, a urology-specific chatbot integrating retrieval-augmented generation (RAG) to provide in-line references and source text previews for each response. In a randomized controlled reader study, UroBot and ChatGPT were compared across ten uro-oncological case rounds. Thirty urologists assessed recommendation correctness, source verifiability and trust with preference ratings collected after each round.

UroBot performed significantly better than ChatGPT in recommendation correctness (73 % vs. 50 %; p < 0.001), source attribution (74 % vs. 30 %; p < 0.001) and verifiability of sources (84 % vs. 35 %; p < 0.001). Furthermore, clinicians consistently preferred UroBot for accuracy, source verifiability and trust. Qualitative analysis showed that ChatGPT often produced vague or incorrect citations, with 28 % being non-existent or outdated and 83 % lacking specific sections, whereas UroBot achieved complete alignment on guideline sub-section and page level. These gains in citation precision were mirrored by higher clinician ratings for verifiability and trust. Limitations include the small sample size of ten cases due to feasibility, which may not cover the full uro-oncological spectrum.

Our findings show that combining LLMs with RAG with in-line references and source text previews markedly enhances perceived source attribution and verifiability compared to state-of-the-art conventional LLMs. Importantly, this approach is readily transferable across medical subspecialties, enabling reliable and up-to-date clinical decision support.