Edge stability and edge quantum criticality in 2D interacting topological insulators

Robustness of helical edge states in 2D topological insulators (TI) against strong interactions remains an intriguing issue. Here, by performing the first sign-free quantum Monte Carlo (QMC) simulation of the Kane-Mele-Hubbard-Rashba model which describes an interacting 2D TI with two-particle backscattering on edges, we verify that the gapless helical edge states are robust against a finite range of two-particle backscattering when the Coulomb repulsion is not strong. However, when the Coulomb repulsion is strong enough, the helical edge states can be gapped by infinitesimal two-particle backscattering, resulting in edge magnetic order. We further reveal universal properties of the magnetic edge quantum critical point (EQCP). At magnetic domain walls on edges, we find that a fractionalized charge of e/2 emerges. Implications of our results to recent transport experiments in the InAs/GaSb quantum well, which is a 2D TI with strong interactions, will also be discussed.