We study spontaneously vectorized black holes in Einstein-vector-Gauss-Bonnet theory with a quadratic coupling function. Besides the static, spherically symmetric black holes carrying an electric charge, there are uncharged static, axially symmetric black holes that possess a magnetic dipole moment. Both types possess radial excitations. The magnetic black holes are prolate. They are hotter than the Schwarzschild black holes and possess lower free energy. The domain of existence of the rotating vectorized black holes is bounded by the Kerr black holes, the spherically and axially symmetric static black holes, and the critical solutions.