Dihedral symmetry in $SU(N)$ Yang-Mills theory

We point out that charge conjugation and coordinate reflection symmetries do not commute with the center symmetry of $SU(N)$ YM theory when $N>2$. As a result, for generic values of the $\theta$ angle, the group of discrete zero-form symmetries of YM theory on e.g. the spacetime manifold $\mathbb{R}^3\times S^1$ includes the dihedral group $D_{2N}$ which is non-Abelian for $N>2$. At $\theta = \pi$, the non-Abelian factor in the symmetry group is enhanced to $D_{4N}$ due to discrete 't Hooft anomaly considerations. We illustrate these results in YM theory as well as in a simple quantum mechanical model, where we study representation theory as a function of $\theta$ angle.