Lattices in Tate modules

Refining a theorem of Zarhin, we prove that given a $g$-dimensional abelian variety $X$ and an endomorphism $u$ of $X$, there exists a matrix $A \in \operatorname{M}_{2g}(\mathbb{Z})$ such that each Tate module $T_\ell X$ has a $\mathbb{Z}_\ell$-basis on which the action of $u$ is given by $A$, and similarly for the covariant Dieudonn\'e module tensored with $\mathbb{Q}$ if over a perfect field of characteristic $p$.