Modular symmetry anomaly and non-perturbative neutrino mass terms in magnetized orbifold models

We study the modular symmetry anomaly in magnetized orbifold models. The non-perturbative effects such as D-brane instanton effects can break tree-level symmetry. We study which part of the modular symmetry is broken explicitly by Majorana mass terms with three generations of neutrinos. The modular weight of neutrino mass terms does not match with other coupling terms in tree-level Lagrangian. In addition, the $Z_N$ symmetry of the modular flavor symmetry is broken and a certain normal subgroup of the modular flavor symmetry remains in neutrino mass terms.