Modular forms and hierarchical Yukawa couplings in heterotic Calabi-Yau compactifications

We study the modular symmetry in heterotic string theory on Calabi-Yau threefolds. In particular, we examine whether moduli-dependent holomorphic Yukawa couplings are described by modular forms in the context of heterotic string theory with standard embedding. We find that $SL(2,\mathbb{Z})$ modular symmetry emerges in asymptotic regions of the Calabi-Yau moduli space. The instanton-corrected holomorphic Yukawa couplings are then given by modular forms under $SL(2,\mathbb{Z})$ or its congruence subgroups such as $\Gamma_0(3)$ and $\Gamma_0(4)$. In addition to the modular symmetry, it turns out that another coupling selection rule controls the structure of holomorphic Yukawa couplings. Furthermore, the coexistence of both the positive and negative modular weights for matter fields leads to a hierarchical structure of matter field K\"ahler metric. Thus, these holomorphic modular forms and the matter field K\"ahler metric play an important role in realizing a hierarchical structure of physical Yukawa couplings.