Holomorphic anomaly equations for the Hilbert scheme of points of a K3 surface

We conjecture that the generating series of Gromov-Witten invariants of the Hilbert schemes of $n$ points on a K3 surface are quasi-Jacobi forms and satisfy a holomorphic anomaly equation. We prove the conjecture in genus $0$ and for at most $3$ markings - for all Hilbert schemes and for arbitrary curve classes. In particular, for fixed $n$, the reduced quantum cohomologies of all hyperk\"ahler varieties of $K3^{[n]}$-type are determined up to finitely many coefficients. As an application we show that the generating series of $2$-point Gromov-Witten classes are vector-valued Jacobi forms of weight $-10$, and that the fiberwise Donaldson-Thomas partition functions of an order two CHL Calabi-Yau threefold are Jacobi forms.