Rank 2 local systems and abelian varieties II

Let $X/\mathbb{F}_{q}$ be a smooth, geometrically connected, quasiprojective variety. Let $\mathcal{E}$ be a semisimple overconvergent $F$-isocrystal on $X$. Suppose that irreducible summands $\mathcal{E}_i$ of $\mathcal E$ have rank 2, determinant $\bar{\mathbb{Q}}_p(-1)$, and infinite monodromy at $\infty$. Suppose further that for each closed point $x$ of $X$, the characteristic polynomial of $\mathcal{E}$ at $x$ is in $\mathbb{Q}[t]\subset \mathbb Q_p[t]$. Then there exists a non-trivial open set $U\subset X$ such that $\mathcal{E}|_U$ comes from a family of abelian varieties on $U$. As an application, let $L_1$ be an irreducible lisse $\bar{\mathbb{Q}}_l$ sheaf on $X$ that has rank 2, determinant $\bar{\mathbb{Q}}_l(-1)$, and infinite monodromy at $\infty$. Then all crystalline companions to $L_1$ exist (as predicted by Deligne's crystalline companions conjecture) if and only if there exists a non-trivial open set $U\subset X$ and an abelian scheme $\pi_U\colon A_U\rightarrow U$ such that $L_1|_U$ is a summand of $R^1(\pi_U)_*\bar{\mathbb{Q}}_l$.