Primordial magnetic fields could provide an explanation for the galactic magnetic fields observed today, in which case they may also leave interesting signals in the CMB and the small-scale matter power spectrum. We discuss how to approximately calculate the important non-linear magnetic effects within the guise of linear perturbation theory, and calculate the matter and CMB power spectra including the SZ contribution. We then use various cosmological datasets to constrain the form of the magnetic field power spectrum. Using solely large-scale CMB data (WMAP7, QUaD and ACBAR) we find a 95% CL on the variance of the magnetic field at 1 Mpc of B_\lambda < 6.4 nG. When we include SPT data to constrain the SZ effect, we find a revised limit of B_\lambda < 4.1 nG. The addition of SDSS Lyman-alpha data lowers this limit even further, roughly constraining the magnetic field to B_\lambda < 1.3 nG.