Distillation with sublogarithmic overhead

It has been conjectured [1] that for any distillation protocol for magic states for the $T$ gate, the number of noisy input magic states required per output magic state at output error rate $\epsilon$ is $\Omega(\log(1/\epsilon))$. We show that this conjecture is false. We find a family of quantum error correcting codes of parameters $[[\sum_{i=w+1}^m \binom{m}{i}, \sum_{i=0}^{w} \binom{m}{i}, \sum_{i=w+1}^{r+1} \binom{r+1}{i}]]$ for any integers $m > 2r$, $r > w \ge 0$, by puncturing quantum Reed-Muller codes. When $m > \nu r$, our code admits a transversal logical gate at the $\nu$-th level of Clifford hierarchy. In a distillation protocol for magic states at the level $\nu = 3$ ($T$-gate), the ratio of input to output magic states is $O(\log^\gamma (1/\epsilon))$ where $\gamma = \log(n/k)/\log(d)< 0.678$ for some $m,r,w$. The smallest code in our family for which $\gamma < 1$ is on $\approx 2^{58}$ qubits.