Descent properties for an abelian variety with extended Galois representation

Let $K$ be a field, $L$ a finite Galois extension of $K$, and $X$ an abelian variety defined over $L$. If $X$ is isogenous over $L$ to an abelian variety defined over $K$, then the $\ell$-adic Galois representations associated to $X$ extend to representations $\bar{\rho}_{\ell,X}:\mathrm{Gal}(\bar{L}/K)\to\mathrm{Aut}(V_\ell X)$ for every prime $\ell$. This paper aims to show that the converse is true for abelian varieties of Type I, with some supplementary conditions needed on the endomorphisms of $X$, when $L$ is either a number field or a function field of prime characteristic different from $2$.