Directionally Locked Heteroepitaxy with a Structurally Modulated van der Waals Material

Precise orientation of symmetry-mismatched epilayers on van der Waals (vdW) substrates via heteroepitaxy has commonly been achieved through surface treatment processes to accommodate weak interlayer registry and bonding strength, thereby limiting the range of material combinations for heterostructure design. In this study, we investigate the influence of lattice instabilities in a structurally modulated vdW TaCo₂Te₂ substrate on the growth and alignment of a symmetry-mismatched bulk Co_xTe_y epilayer using in situ heating in a transmission electron microscope (TEM). We show that a Peierls-like lattice instability occurs in TaCo₂Te₂ at a transition temperature of ∼523 K, which was corroborated by phonon calculations. Postheat-treated samples reveal a thermally induced surface diffusion process and the dominant lateral growth of the Co_xTe_y epilayer on the TaCo₂Te₂ vdW layers, as observed in cross-sectional TEM images. Temperature-dependent selected area electron diffraction (SAED) patterns reveal that the quasi-vdW Co_xTe_y/TaCo₂Te₂ heterointerface acquires directional locking by aligning larger interlayer lattice mismatch along the lattice instability axis of TaCo₂Te₂, while preserving a strong lattice matching along the orthogonal direction. This heterostructure exhibits precise interlayer registry with one-dimensional lattice incommensuration along the lattice instability axis, resulting from structural distortion to accommodate lattice-mismatch strain. Moreover, the interfacial reconstruction of TaCo₂Te₂ back to the distorted phase stabilizes the lattice-locking of the quasi-vdW heterointerface at elevated temperatures. These findings encourage the expansion of material diversity for designing and predicting multidimensional heterostructures by leveraging lattice instabilities to guide epitaxy.