Negative powers of Hilbert-space contractions

We show that, given a closed subset $E$ of the unit circle of Lebesgue measure zero, there exists a positive sequence $u_n\to\infty$ with the following property: if $T$ is a Hilbert-space contraction such that $\sigma(T)\subset E$ and $\|T^{-n}\|=O(u_n)$ and rank$(I-T^*T)<\infty$, then $T$ is a unitary operator. We further show that the condition of measure zero is sharp.