Ordinary primes for $\mathrm{GL}_2$-type abelian varieties and weight $2$ modular forms

Let $A$ be a $g$-dimensional abelian variety defined over a number field $F$. It is conjectured that the set of ordinary primes of $A$ over $F$ has positive density, and this is known to be true when $g=1, 2$, or for certain abelian varieties with extra endomorphisms. In this paper, we extend the family of abelian varieties whose sets of ordinary primes have positive density. Specifically, we show that if the endomorphism algebra of $A$ contains a number field $K$ of degree $g$, then under certain conditions on the fields $F$ and $K$, the set of ordinary primes of $A$ over $F$ has positive density. This includes $\mathrm{GL}_2$-type abelian varieties over $\mathbb{Q}$ (resp. quadratic number fields) of dimension $q$ or $2q$ (resp. $q$) for any rational prime $q$. The proof is carried out in the general setting of compatible systems of Galois representations, and as a consequence, it also implies a positive density result for the sets of ordinary primes of certain modular forms of weight $2$.