The $\mathfrak{sl}_\infty$-crystal combinatorics of higher level Fock spaces

For integers $e,\ell\geq 2$, the level $\ell$ Fock space has an $\mathfrak{sl}_\infty$-crystal structure arising from the action of a Heisenberg algebra, intertwining the $\widehat{\mathfrak{sl}_e}$-crystal. The vertices of these crystals are charged $\ell$-partitions. We give the combinatorial rule for computing the arrows anywhere in the $\mathfrak{sl}_\infty$-crystal. This allows us to pinpoint the location of any charged $\ell$-partition. As an application, we compute the support of the spherical representation of a cyclotomic rational Cherednik algebra, and in particular, the set of parameters such that it is finite-dimensional. We also give an easy abacus characterization of all finite-dimensional representations of type $B$ Cherednik algebras.