Atomic-Scale Magnetism by Embedded Co Atoms on the Surface of a Topological Insulator

Topological insulators (TIs) represent an electronic phase characterized by spin-momentum-locked topological surface states (TSSs) that span the bulk bandgap. These states enable chiral electronic channels with perfect conduction, which are promising for applications ranging from quantum information processing to dissipationless charge and spin transport. However, the surfaces of TIs typically host gapless states protected by time-reversal symmetry (TRS). Introducing magnetic impurities into TIs breaks TRS, which is a prerequisite for realizing phenomena like the topological magnetoelectric effect. Nevertheless, the role of atomic-scale magnetism in magnetic TIs remains poorly understood. Here, we provide detailed insight into atomic-scale magnetism induced in the Bi₂Te₃ TI by exclusively doping the top quintuple layer (QL) with the 3d transition metal cobalt (Co). By combining scanning tunneling microscopy/spectroscopy (STM/STS), single-atom deposition, and density functional theory (DFT), we demonstrate the potential to generate a subsurface, atomically thin Co-doping layer without compromising the bulk-insulating properties of the TI. We show that adsorbed Co atoms incorporate substitutionally at Bi sites in the second atomic layer of Bi₂Te₃(111), leading to a spin-polarized electronic configuration with a large out-of-plane magnetic moment. This results from p-d orbital hybridization between the 3d impurity bands and neighboring Te atoms. The Co_Bi substitutions induce spatially localized in-gap states near the magnetic impurities. At larger distances, however, the magnetic proximity effect can still lead to a bandgap opening capable of supporting a quantum anomalous Hall state. Combining STM images and DFT calculations, we find that, following a single Co_Bi substitution, subsequent adsorbed Co atoms are preferentially incorporated adjacent to the initial atom, forming double, triple, and larger Co_Bi structures. The DFT calculations reveal ferromagnetic (FM) coupling between the adjacent Co atoms. Thus, by controlling the density of single Co_Bi substitutions, it is plausible to achieve long-range FM order while preserving the bulk-insulating properties. A possible next step is the formation of embedded Co subsurface 2D islands or even a monolayer, whose electronic properties are naturally protected by the top Te atomic layer. Our findings suggest an alternative doping mechanism focused on the second atomic layer of the top QL─where the TSSs are dominant─thus avoiding the largely undesirable effects of bulk doping.