Tuning insulator-semimetal transitions in 3D topological insulator thin films by inter-surface hybridization and in-plane magnetic fields

A pair of Dirac points (analogous to a vortex-antivortex pair) associated with opposite topological numbers (with $\pm\pi$ Berry phases) can be merged together through parameter tuning and annihilated to gap the Dirac spectrum, offering a canonical example of a topological phase transition. Here, we report transport studies on thin films of BiSbTeSe$_2$ (BSTS), which is a 3D TI that hosts spin-helical gapless (semi-metallic) Dirac fermion surface states (SS) for sufficiently thick samples, with an observed resistivity close to $h/4e^2$ at the charge neutral point. When the sample thickness is reduced to $\sim$10 nm thick, the Dirac cones from the top and bottom surfaces can hybridize (analogous to a "merging" in the real space) and become gapped to give a trivial insulator. Furthermore, we observe that an in-plane magnetic field can drive the system again towards a metallic behavior, with a prominent negative magnetoresistance (MR, up to $\sim$$-$95\%) and a temperature-insensitive resistivity close to $h/2e^2$ at the charge neutral point. The observation is interpreted in terms of a predicted effect of an in-plane magnetic field to reduce the hybridization gap (which, if small enough, may be smeared by disorder and a metallic behavior). A sufficiently strong magnetic field is predicted to restore and split again the Dirac points in the momentum space, inducing a distinct 2D topological semimetal (TSM) phase with 2 single-fold Dirac cones of opposite spin-momentum windings.