LLM Detoxification via Transition-Aware Reward-Guided Representation Sampling

Although Large Language Models (LLMs) have showcased remarkable capabilities across diverse tasks, their susceptibility to producing harmful or toxic content underscores the urgent need for effective detoxification strategies to ensure their safe and ethical deployment. Test-time detoxification methods, which often involve directional steering within LLM representations during inference, have emerged as one of the most promising approaches due to their flexibility and minimal invasiveness. However, current approaches often rely on static or coarse interventions and suffer from imprecise guidance, largely due to their limited ability to model the transition process between toxic and non-toxic generations. To address this challenge, we propose Transition-Aware Reward-Guided Representation Sampling (TARGRS), a test-time detoxification framework that learns reward guidance directly over toxicity transitions in the latent representation space. Rather than relying on sparse toxic or non-toxic representations, our method performs progressive directional steering during generation to trace representation trajectories that span the transition from non-toxic to toxic states. Each trajectory is assigned toxicity scores, converting sparse annotations into dense, listwise supervision over fine-grained transitions. Based on this formulation, we train a token-level reward model whose predictions are explicitly transition-aware, enabling accurate control over representation sampling during inference. During exploration, steering along the toxicity direction constructs transition trajectories for reward learning; during inference, TARGRS performs coarse-to-fine sampling along the opposite, non-toxic direction to obtain detoxified representations. Specifically, coarse sampling first identifies the intervention required to reach non-toxic regions, followed by refinement to select the optimal representation with minimal perturbation. Comprehensive experiments on 10 widely used LLMs across the RealToxicityPrompts, BOLD, and AttaQ benchmarks demonstrate that TARGRS achieves consistent and strong detoxification performance, significantly outperforming state-of-the-art baselines. For instance, TARGRS reduces toxicity by up to 69.38 Additionally, TARGRS preserves the model's core capabilities with minimal impact. Furthermore, benefiting from intervention in the language decoder space, TARGRS can be seamlessly extended to Large Vision-Language Models (LVLMs), where it also demonstrates strong effectiveness, reducing toxicity by up to 64.23 https://github.com/xiaoyisong/TARGRS.