Trading inverses for an irrep in the Solovay-Kitaev theorem

The Solovay-Kitaev theorem states that universal quantum gate sets can be exchanged with low overhead. More specifically, any gate on a fixed number of qudits can be simulated with error $\epsilon$ using merely $\mathrm{polylog}(1/\epsilon)$ gates from any finite universal quantum gate set $\mathcal{G}$. One drawback to the theorem is that it requires the gate set $\mathcal{G}$ to be closed under inversion. Here we show that this restriction can be traded for the assumption that $\mathcal{G}$ contains an irreducible representation of any finite group $G$. This extends recent work of Sardharwalla et al. [arXiv:1602.07963], and applies also to gates from the special linear group. Our work can be seen as partial progress towards the long-standing open problem of proving an inverse-free Solovay-Kitaev theorem [arXiv:quant-ph/0505030, arXiv:0908.0512].