Rank 2 sheaves on toric 3-folds: classical and virtual counts

Let $\mathcal{M}$ be the moduli space of rank 2 stable torsion free sheaves with Chern classes $c_i$ on a smooth 3-fold $X$. When $X$ is toric with torus $T$, we describe the $T$-fixed locus of the moduli space. Connected components of $\mathcal{M}^T$ with constant reflexive hulls are isomorphic to products of $\mathbb{P}^1$. We mainly consider such connected components, which typically arise for any $c_1$, "low values" of $c_2$, and arbitrary $c_3$. In the classical part of the paper, we introduce a new type of combinatorics called double box configurations, which can be used to compute the generating function $\mathsf{Z}(q)$ of topological Euler characteristics of $\mathcal{M}$ (summing over all $c_3$). The combinatorics is solved using the double dimer model in a companion paper. This leads to explicit formulae for $\mathsf{Z}(q)$ involving the MacMahon function. In the virtual part of the paper, we define Donaldson-Thomas type invariants of toric Calabi-Yau 3-folds by virtual localization. The contribution to the invariant of an individual connected component of the $T$-fixed locus is in general not equal to its signed Euler characteristic due to $T$-fixed obstructions. Nevertheless, the generating function of all invariants is given by $\mathsf{Z}(q)$ up to signs.