Evolution from B_{2g} Nematics to B_{1g} Nematics in Heavily Hole-Doped Iron-Based Superconductors

Recent experiments reported an unusual nematic behavior of heavily hole-doped pnictides AFe_{2}As_{2}, with alkali A=Rb, Cs. In contrast to the B_{2g} nematic order of the parent AeFe_{2}As_{2} compounds (with alkaline earth Ae=Sr, Ba), characterized by unequal nearest-neighbor Fe-Fe bonds, in the hole-doped systems nematic order is observed in the B_{1g} channel, characterized by unequal next-nearest-neighbor Fe-Fe (diagonal Fe-As-Fe) bonds. In this Letter, using density functional theory, we attribute this behavior to the evolution of the magnetic ground state along the series Ae_{1-x}A_{x}Fe_{2}As_{2}, from single stripes for small x to double stripes for large x. Our simulations using the reduced Stoner theory show that fluctuations of Fe moments are essential for the stability of the double-stripe configuration. We propose that the change in the nature of the magnetic ground state is responsible for the change in the symmetry of the vestigial nematic order that it supports.