Unlikely intersections with $E$ $\times$ CM curves in $\mathcal{A}_2$

The Zilber-Pink conjecture predicts that an algebraic curve in $\mathcal{A}_2$ has only finitely many intersections with the special curves, unless it is contained in a proper special subvariety. Under a large Galois orbits hypothesis, we prove the finiteness of the intersection with the special curves parametrising abelian surfaces isogenous to the product of two elliptic curves, at least one of which has complex multiplication. Furthermore, we show that this large Galois orbits hypothesis holds for curves satisfying a condition on their intersection with the boundary of the Baily-Borel compactification of $\mathcal{A}_2$. More generally, we show that a Hodge generic curve in an arbitrary Shimura variety has only finitely many intersection points with the generic points of a Hecke-facteur family, again under a large Galois orbits hypothesis.