Magnetic Orders Proximal to the Kitaev Limit in Frustrated Triangular Systems: Application to Ba$_3$IrTi$_2$O$_9$

Frustrated transition metal compounds in which spin-orbit coupling (SOC) and electron correlation work together have attracted much attention recently. In the case of 5$d$ transition metals, where SOC is large, $j_\text{eff}=1/2$ bands near the Fermi level are thought to encompass the essential physics of the material, potentially leading to a concrete realization of exotic magnetic phases such as the Kitaev spin liquid. Here we derive a spin model on a triangular lattice based on $j_\text{eff} = 1/2$ pseudospins that interact via antiferromagnetic Heisenberg ($J$) and Kitaev ($K$) exchanges, and crucially, an anisotropic $(\Gamma)$ exchange. Our classical analysis of the spin model reveals that, in addition to small regions of 120$^\circ$, $\mathbb{Z}_2$ / dual-$\mathbb{Z}_2$ vortex crystal and nematic phases, the stripy and ferromagnetic phases dominate the $J$-$K$-$\Gamma$ phase diagram. We apply our model to the 5$d$ transition metal compound, Ba$_3$IrTi$_2$O$_9$, in which the Ir$^{4+}$ ions form layered two-dimensional triangular lattices. We compute the band structure and nearest-neighbor hopping parameters using ab-initio calculations. By combining our ab-initio and classical analyses, we predict that Ba$_3$IrTi$_2$O$_9$ has a stripy ordered magnetic ground state.