Absence of long-range order in a triangular spin system with dipolar interactions

Antiferromagnetic Heisenberg model on the triangular lattice is perhaps the best known example of frustrated magnets, but it orders at low temperatures. Recent density matrix renormalization group (DMRG) calculations find that next nearest neighbor interaction $J_2$ enhances the frustration and leads to a spin liquid for $J_2/J_1\in (0.08,0.15)$. In addition, DMRG study of a dipolar Heisenberg model with longer range interactions gives evidence for a spin liquid at small dipole titling angle $\theta\in[0,10^\circ)$. In both cases, the putative spin liquid region appears to be small. Here, we show that for the triangular lattice dipolar Heisenberg model, a robust quantum paramagnetic phase exists in a surprisingly wide region, $\theta\in [0,54^\circ)$, for dipoles tilted along the lattice diagonal direction. We obtain the phase diagram of the model by functional renormalization group (RG) which treats all magnetic instabilities on equal footing. The quantum paramagnetic phase is characterized by a smooth continuous flow of vertex functions and spin susceptibility down to the lowest RG scale, in contrast to the apparent breakdown of RG flow in phases with stripe or spiral order. Our finding points to a promising direction to search for quantum spin liquids in ultracold dipolar molecules.