Faithful Explanation Regeneration via Cauchy-Schwarz Mixture Information Bottleneck

Large pretrained language models can generate fluent free-text explanations for natural language reasoning tasks, but these explanations may contain redundant, irrelevant, or unsupported information. In this paper, we propose a faithful explanation regeneration framework based on a Cauchy-Schwarz mixture information bottleneck. The proposed method compresses noisy explanations into a structured bottleneck representation while preserving task-relevant and decision-supporting information. Instead of using a unimodal Gaussian prior, we introduce a Gaussian mixture prior and employ the Cauchy-Schwarz divergence as a tractable compression regularizer. Furthermore, a faithfulness-aware objective is introduced to encourage the learned representation to remain aligned with the task decision. Experiments on free-text explanation benchmarks demonstrate that the proposed method improves explanation quality, conciseness, and faithfulness while providing an information-theoretic compression-preservation framework.