Optimization of Grand-Canonical Self-Consistent Field across Manifolds with Varying Occupation Spaces

Self-consistent-field (SCF) in the grand-canonical (GC) ensemble faces convergence difficulties with significant fractional occupation at low temperatures. By recognizing the orbital coefficient matrix and the fractional occupation vectors as two independent variables, this work provides a new viewpoint upon GC-SCF as an optimization problem on a product manifold of a flag manifold and a Euclidean space. During the optimization process, the manifold is automatically adjusted to the orbitals, which are divided into three partitions based on the occupation numbers, so that the occupation numbers are optimized properly, avoiding notorious gradient explosion and vanishing. Important concepts in manifold optimization are discussed and their specific expressions in GC-SCF are given. Via numerical benchmarks on various examples, our algorithms are shown to be more efficient than the conventional direct inversion in the iterative subspace (DIIS). Among them, we recommend the augmented Roothaan-Hall method, which reaches the balance between time consumption and convergence rate.