Helical structures in layered magnetic superconductors due to indirect exchange interactions mediated by interlayer tunneling

Motivated by the recent discovery of helical magnetic structure in RbEuFe$_{4}$As$_{4}$, we investigate interlayer ordering of magnetic moments in materials composed of spatially-separated superconducting and ferromagnetically-aligned layers. We consider the interplay between the normal and superconducting indirect exchange interaction mediated by tunneling between the conducting layers. We elaborate a recipe to evaluate the normal interlayer interaction via two-dimensional density of states of an isolated layer and demonstrate that for bands with small fillings, such interaction is typically ferromagnetic and short-range. The nearest-layer interaction is proportional to the ratio of the interlayer hopping and in-plane band width squared. On the other hand, the superconducting contribution always gives antiferromagnetic interaction and may extend over several layers when the interlayer hopping energy exceeds the superconducting gap. The frustration caused by the interplay between the normal and superconducting parts may lead to spiral ground-state magnetic configuration. The four-fold in-plane anisotropy may lock the rotation angle between the moments in the neighboring layers to $90^\circ$, as it was observed in RbEuFe$_{4}$As$_{4}$