What determines the ultimate precision of a quantum computer?

A quantum error correction (QEC) code uses $N_{\rm c}$ quantum bits to construct one "logical" quantum bits of better quality than the original "physical" ones. QEC theory predicts that the failure probability $p_L$ of logical qubits decreases exponentially with $N_{\rm c}$ provided the failure probability $p$ of the physical qubit is below a certain threshold $p<p_{\rm th}$. In particular QEC theorems imply that the logical qubits can be made arbitrarily precise by simply increasing $N_{\rm c}$. In this letter, we search for physical mechanisms that lie outside of the hypothesis of QEC theorems and set a limit $\eta_{\rm L}$ to the precision of the logical qubits (irrespectively of $N_{\rm c}$). $\eta_{\rm L}$ directly controls the maximum number of operations $\propto 1/\eta_{\rm L}^2$ that can be performed before the logical quantum state gets randomized, hence the depth of the quantum circuits that can be considered. We identify a type of error - silent stabilizer failure - as a mechanism responsible for finite $\eta_{\rm L}$ and discuss its possible causes. Using the example of the topological surface code, we show that a single local event can provoke the failure of the logical qubit, irrespectively of $N_c$.