S-duality and loop operators in canonical formalism

We study the gauge invariant 't Hooft operator in canonical formalism for Yang-Mills theory as well as the $\mathcal{N} =4$ super-Yang-Mills theory with the gauge group $U(N)$. It is shown that the spectrum of the 't Hooft operator labeled by the arbitrary irreducible representation of the gauge group is the same as the spectrum of the Wilson operator labeled by the same representation. So it is possible to construct a unitary operator $S$ making the two kinds of loop operators transformed into each other. S-duality transformation could be realized by the operator $S$. We compute the supersymmetry variations of the loop operators with the fermionic couplings turned off. The result is consistent with the expectation that the action of $S$ should make supercharges transform with a $U(1)_{Y}$ phase.