Infinitely many Shimura varieties in the Jacobian locus for $g \leq 4$

We study families of Galois covers of curves of positive genus. It is known that under a numerical condition these families yield Shimura subvarieties generically contained in the Jacobian locus. We prove that there are only 6 families satisfying this condition, all of them in genus 2,3 or 4. We also show that these families admit two fibrations in totally geodesic subvarieties, generalizing a result of Grushevsky and M\"oller. Countably many of these fibres are Shimura. Thus the Jacobian locus contains infinitely many Shimura subvarieties of positive dimension of any $g \leq 4$.