Mod-$p$ Galois representations not arising from abelian varieties

It is known that any Galois representation $\rho : G_{\mathbb{Q}} \rightarrow \mathrm{GL}(2,\mathbb{F}_p)$ with determinant equal to the mod-$p$ cyclotomic character, arises from the $p$-torsion of an elliptic curve over $\mathbb{Q}$, if and only if $p \leq 5$. In dimension $g = 2$, when $p \le 3$, it is again known that any Galois representation valued in $\mathrm{GSp}(4,\mathbb{F}_p)$ with cyclotomic similitude character arises from an abelian surface. In this paper, we study this question for all primes $p$ and dimensions $g \ge 2$. When $g \ge 2$ and $(g,p) \neq (2,2)$, $(2,3)$, $(3,2)$, we prove the existence of a Galois representation over $\mathbb{Q}$ valued in $\mathrm{GSp}(2g,\mathbb{F}_p)$ with cyclotomic similitude character, that cannot arise as the $p$-torsion representation of any $g$-dimensional abelian variety over $\mathbb{Q}$.