Three-dimensional imprimitive representations of the modular group and their associated modular forms

This paper uses previous results of the authors on vector-valued modular forms to study certain non-congruence modular forms. We prove that these forms have unbounded denominators, and in certain cases we verify congruences of Atkin--Swinnerton-Dyer type satisfied by the Fourier coefficients of these forms. Our results rest on group-theoretic facts about the modular group, a detailed study of its imprimitive three-dimensional representations, and the theory of their associated vector-valued modular forms. For the proof of the congruences we also make essential use of a result of Katz.