Irreducibility of polarized automorphic Galois representations in infinitely many dimensions

Let $\pi$ be a polarized, regular algebraic, cuspidal automorphic representation of $\operatorname{GL}_n(\mathbb{A}_F)$ where $F$ is totally real or imaginary CM, and let $(\rho_\lambda)_\lambda$ be its associated compatible system of Galois representations. Suppose that $7\nmid n$ and, if $4\mid n$, then $n = 4p$ for some prime number $p$. We prove that there is a Dirichlet density $1$ set of rational primes $\mathcal{L}$ such that whenever $\lambda\mid \ell$ for some $\ell\in \mathcal{L}$, then $\rho_\lambda$ is irreducible.