Compatibility of canonical $\ell$-adic local systems on Shimura varieties

For a Shimura variety $(G, X)$ in the superrigid regime and neat level subgroup $K_0$, we show that the canonical family of $\ell$-adic representations associated to a number field point $y \in \mathrm{Sh}_{K_0}(G, X)(F)$, $$\left\{ \rho_{y, \ell} \colon \mathrm{Gal}(\overline{\mathbb{Q}}/F) \to G^{\mathrm{ad}}(\mathbb{Q}_{\ell}) \right\}_{\ell},$$ form a compatible system of $G^{\mathrm{ad}}(\mathbb{Q}_{\ell})$-representations: there is an integer $N(y)$ such that for all $\ell$, $\rho_{y, \ell}$ is unramified away from $N(y) \ell$, and for all $\ell \neq \ell'$ and $v \nmid N(y)\ell \ell'$, the semisimple parts of the conjugacy classes of $\rho_{y, \ell}(\mathrm{Frob}_v)$ and $\rho_{y, \ell'}(\mathrm{Frob}_v)$ are ($\mathbb{Q}$-rational and) equal. We deduce this from a stronger compatibility result for the canonical $G(\mathbb{Q}_{\ell})$-valued local systems on connected Shimura varieties inside $\mathrm{Sh}_{K_0}(G, X)$. Our theorems apply in particular to Shimura varieties of non-abelian type and represent the first such independence-of-$\ell$ results in non-abelian type.