Testing accuracy of qubit rotations on a public quantum computer

We analyze the results of the test of $\pi/2$ qubit rotations on the public quantum computer provided by IBM. We measure a single qubit rotated by $\pi/2$ about a random axis, and we accumulate vast statistics of the results. The test performed on different devices shows systematic deviations from the theoretical predictions, which appear at the level $10^{-3}$. Some of the differences, beyond 5 standard deviations, cannot be explained by simple corrections due to nonlinearities of pulse generations. The magnitude of the deviation is comparable with the randomized benchmarking of the gate, but we additionally observe a pronounced parametric dependence. We discuss other possible reasons of the deviations, including states beyond the single-qubit space. The deviations have a similar structure for various devices used at different times, and so they can also serve as a diagnostic tool to eliminate imperfect gate implementations, and faithful description of the involved physical systems.