Singular Gauss sums, Polya-Vinogradov inequality for $GL(2)$ and growth of primitive elements

We establish an analogue of the classical Polya-Vinogradov inequality for $GL(2, \F_p)$, where $p$ is a prime. In the process, we compute the `singular' Gauss sums for $GL(2, \F_p)$. As an application, we show that the collection of elements in $GL(2,\Z)$ whose reduction modulo $p$ are of maximal order in $GL(2, \F_p)$ and whose matrix entries are bounded by $x$ has the expected size as soon as $x\gg p^{1/2+\ep}$ for any $\ep>0$. In particular, there exist elements in $GL(2,\Z)$ with matrix entries that are of the order $O(p^{1/2+\ep})$ whose reduction modulo $p$ are primitive elements.