Physics-informed transformers for electronic quantum states

Neural-network-based variational quantum states, particularly autoregressive models, are powerful tools for describing complex many-body wave functions. However, their performance depends on the computational basis chosen and they often lack physical interpretability. We propose a modified variational Monte-Carlo framework which leverages prior physical information to construct a complete computational many-body basis containing a reference state that serves as a rough approximation to the true ground state. A Transformer is used to parametrize and autoregressively sample corrections to this reference state, giving rise to a more interpretable and computationally efficient representation of the ground state. We demonstrate this approach in a fermionic model featuring a metal-insulator transition by employing Hartree-Fock and a strong-coupling limit to define physics-informed bases. We also show that the Transformer's hidden representation captures the natural energetic order of the different basis states. This work paves the way for more efficient and interpretable neural quantum-state representations.