Why do non-gauge invariant terms appear in the vacuum polarization tensor?

It is will known that quantum field theory at the formal level is gauge invariant. However a calculation of the vacuum polarization tensor will include non-gauge invariant terms. These terms must be removed from the calculation in order to get a physically correct result. One common way to do this today is the technique of "dimensional regularization". It has recently been noted [2] that at one time a supersymmetric-like solution to the problem was explored - that is, the right combination of fields would cause the offending terms to cancel out. I will examine some of this early work and pose the question - why do the non-gauge invariant terms appear in the first place? I will show that this is due to an improper mathematical step in the formulation of the pertubative expansion. I will then show that when this step is corrected the result is gauge invariant. However a supersymmetric-like solution is still required to cancel out a divergent term.