Mott transition and magnetism on the anisotropic triangular lattice

Spin-liquid behavior was recently suggested experimentally in the moderately one-dimensional organic compound $\kappa$-H$_3$(Cat-EDT-TTF)$_2$. This compound can be modeled by the one-band Hubbard model on the anisotropic triangular lattice with $t^\prime/t \simeq 1.5$, where $t'$ is the minority hopping. It thus becomes important to extend previous studies, that were performed in the range $0 \leq t'/t \leq 1.2$, to find out whether there is a regime where Mott insulating behavior can be found without long-range magnetic order. To this end, we study the above model in the range $1.2 \leq t'/t \leq 2$ using cluster dynamical mean-field theory (CDMFT). We argue that it is important to choose a symmetry-preserving cluster rather than a quasi one-dimensional cluster. We find that, upon increasing $t'/t$ beyond $t^\prime/t \approx 1.3$, the Mott transition at zero-temperature is replaced by a first-order transition separating a metallic state from a collinear magnetic insulating state. Nevertheless, at the physically relevant value $t^\prime/t \simeq 1.5$, the transitions toward the magnetic and the Mott insulating phases are very close. The phase diagram obtained in this study can provide a working basis for moderately one-dimensional compounds on the anisotropic triangular lattice.