Researchers at Dept. of Physics, Southern University of Science and Technology (SUSTech) have made significant progress in magnetic topological materials. The scientists elucidate an unprecedented hybridization effect between the building blocks of the topological surface states and reveal the termination-dependent time-reversal symmetry breaking in a magnetic topological insulator.

Department of Physics Associate Professor Yue Zhao and Associate Professor Chang Liu, and Associate Professor Qihang Liu led research groups to publish their significant findings in Physical Review X (IF=12.090), and the paper was entitled “Distinct Topological Surface States on the Two Terminations of MnBi4Te7.”

Figure 1. (a) Schematic diagram of MnBi4Te7 structure. (b) The STM image shows the Bi2Te3 surface and MnBi2Te4 surface after being cleaved. (c)-(f) ARPES experimental results and theoretical calculation results of the energy bands of the two surfaces.

Topological insulators (TIs) have an insulating interior but a metallic surface, at which the electrons are conducting like relativistic particles without dissipation. In magnetic TIs with specific magnetization, the surface state loses most of its metallic nature because of the breaking of some essential symmetries. Thus, introduce magnetic order into TIs will bring possibilities for the realization of novel topological quantum states, such as the quantum anomalous Hall effect. The long-range magnetic order in topological insulator materials is generally achieved by doping magnetic elements. However, magnetic dopants inevitably introduce inhomogeneity and disorder, which is detrimental to observing these novel quantum effects and their realization temperature. Since last year, the discovery of MnBi2Te4(Bi2Te3)n (n = 1, 2, 3, …) series of intrinsic magnetic topological insulators has provided a new platform for this research field.

MnBi4Te7 is composed of alternating magnetic topological insulator MnBi2Te4 layers and non-magnetic topological insulator Bi2Te3 layers, and its natural cleavage surface can be both terminations. We present the first local probe STM measurements of MnBi4Te7 (n = 1) in both real and momentum spaces. Together with the input of ARPES and theoretical calculations, we show that the Bi2Te3 surface displays a gap induced by the hybridization between the bands of the topmost Bi2Te3 layer and the MnBi2Te4 layer beneath, hiding the magnetism induced gap near the Dirac point. In contrast, the MnBi2Te4 surface is gapless due to the magnetically disordered surface.

We also investigated the scattering process by quasi particle interference patterns. The quasi particle interference study clarifies the spin configuration of the topological surface bands and their robustness against scattering. The Bi2Te3 layer carries a spin-helical Dirac state with hexagonal warping, resulting in a quasi-particle interference image similar to the Bi2Te3 crystal surface. In contrast, at the MnBi2Te4 surface, a strongly canted helical state from the surface lies between a pair of Rashba-like splitting bands from its neighboring layer. The hybridization effects between the Rashba splitting bands of the underneath Bi2Te3 layer and the strongly warped band from the surface MnBi2Te4 play an important role in the scattering process.

Figure 2. Schematics of the spin texture, ARPES contours of constant energy (CCE), Joint density of states (JDOS), and spin-selective JDOS calculated from ARPES data and Fourier transferred STS maps (FT-STS) at −50 meV of the two terminations. QL stands for Bi2Te3 quintuple layer; SL stands for MnBi2Te4 septuple layer.

Our results show an unexpected, reversed magnetic properties at the topological surface states for the two terminations of MnBi4Te7. Unlike conventional TI’s, the surface of MnBi4Te7 strongly relies on the hybridization of bands from different building blocks of magnetic TI and non-magnetic TI, in sharp contrast to the conventional topological insulators where the topological surface states strongly localized at the surface layer with attenuation into the bulk. The investigation in both real and momentum spaces provide important information on the scattering process in TI systems. Our findings shed light on the surface bulk correspondence of magnetic topological insulators and reveal the hybridization effects in the heterostructure engineering in the emerging intrinsic magnetic TI systems.

Xuefeng Wu，senior research fellow in Yue Zhao’s group, Jiayu Li, a Ph.D. student in Qihang Liu’s group, Xiaoming Ma, research assistant professor in the Shenzhen Institute for Quantum Science and Engineering (SIQSE), and Zhang Yu, a Ph.D. student in Yue Zhao’s group，are the co-first authors. Qihang Liu, Chang Liu, and Yue Zhao are the corresponding authors. The Department of Physics and the Shenzhen Institute for Quantum Science and Engineering (SIQSE) is the leading affiliation of the article.

This work was supported by the National Natural Science Foundation of China (NSFC), the Key-Area Research and Development Program of Guangdong Province, Guangdong Innovative and Entrepreneurial Research Team Program, the Guangdong Provincial Key Laboratory of Computational Science and Material Design. The research team would like to thank the Hiroshima Synchrotron Radiation Center for their strong support for the work.

Paper Link：https://journals.aps.org/prx/abstract/10.1103/PhysRevX.10.031013