Navigating Oral Cancer Management: Combining Artificial Intelligence and Clinical Guidelines for Optimal...

Decision-making is one of the most difficult and important tasks performed by surgeons. Clinical guidelines and, more recently, Artificial Intelligence (AI) have been implemented to support these decisions. However, there is a paucity of research exploring the impact of AI on surgical decision-making strategies.

To evaluate the accuracy of AI in informing clinical decision-making for the treatment of advanced oral cancer.

Structured questions based on seven selected clinical recommendations were formulated and processed through multiple large language model (LLM) platforms. Claude (Sonnet 3.5) was used to identify discrepancies between established guidelines and LLM-generated responses. Subsequently, three subject matter experts assessed the identified differences to evaluate their clinical significance.

Analysis of the 28 generated responses revealed that 21 (75%) showed congruence between LLM outputs and established guideline recommendations. In three instances, guidelines provided significantly more comprehensive content than LLM responses. One LLM response contained additional relevant information, while three responses were contradictory to the guidelines.

The LLMs demonstrated 75% concordance with guideline recommendations and should be considered complementary tools for established clinical guidelines.