Toward Robust Three-State Diabatization: A Systematic Benchmark of O(3) Transformation Parameterizations

While direct adiabatic-to-diabatic (ATD) transformations for two-state systems have been extensively studied, their extension to three-state systems remains less explored. Building upon the framework of the valence bond wave function-based automatic diabatization (VBADA) approach, this study investigates the application of various O(3) group unitary transformations for the construction of three-state diabatic representations. These include Householder and composite Givens transformations, Euler angles, Tait-Bryan angles, quaternions, and the Rodrigues' rotation formula. Using the dissociation of lithium hydride and the hydrogen abstraction reaction H + HCl → Cl + H₂ as test cases, we evaluate the performance of these different unitary matrices in the diabatization process. Our results reveal a clear performance hierarchy among the O(3) parametrizations. While transformations such as Householder reflections and composite Givens rotations show numerical instability in intermediate geometric regions, and Euler/Tait-Bryan angles are prone to gimbal-lock singularities, both the Rodrigues' rotation formula and quaternion-based parametrizations consistently generate smooth, stable diabatic states with well-localized chemical character.