Rare-earth monopnictides are a family of materials simultaneously displaying complex magnetism, strong electronic correlation, and topological band structure. The recently discovered emergent arc-like surface states in these materials have been attributed to the multi-wave-vector antiferromagnetic order, yet the direct experimental evidence has been elusive. Here we report the observation of non-collinear antiferromagnetic order with multiple modulations using spin-polarized scanning tunneling microscopy. Moreover, we discover a hidden spin-rotation transition of single-to-multiple modulations 2 K below the Neel temperature. The hidden transition coincides with the onset of the surface state splitting observed by our angle-resolved photoemission spectroscopy measurements. Single modulation gives rise to a band inversion with induced topological surface states in a local momentum region while the full Brillouin zone carries trivial topological indices, and multiple modulation further splits the surface bands via non-collinear spin tilting, as revealed by our calculations. The direct evidence of the non-collinear spin order in NdSb not only clarifies the mechanism of the emergent topological surface states but also opens up a new paradigm of control and manipulation of band topology with magnetism.

The STM data was collected by Omicron LT-STM at 4K. MTRX files were exported to the IgorPro files with the software Vernissage. Data analysis was performed on IgorPro. The figures are plotted with Origin and arranged in Adobe Illustrator.

Use Vernissage to open the head file ending with "_0001.mtrx" and inspect the data contained within. The data can be exported as IgorPro files and further analyzed in IgorPro.

IPF files are IgorPro procedures for data analysis.