Delusion Is Not a Bug: AI, Psychiatry, and the Ethics of Uncertainty

Recent debates about AI "guardrails" in mental health have framed a problem as one of detecting delusion. This paper argues that such framing misunderstands both psychiatry and large language models. In clinical practice, delusion is not simply a false belief but a structure of conviction that stabilizes meaning under conditions of uncertainty. Large language models, by contrast, generate coherence rather than evaluate truth. When these systems interact with users seeking orientation, they do not merely fail or succeed at detection-they participate in the conditions under which belief forms. The ethical risk is therefore not misclassification but premature coherence: the subtle reinforcement of interpretations before they can be examined. Drawing on phenomenology and clinical psychiatry, this paper proposes that safety in AI systems should be understood as the protection of provisionality-the preservation of the human capacity to remain uncertain long enough to think.