Large language models (LLMs) based on the Transformer have demonstrated strong performance across diverse tasks. However, current models still exhibit substantial limitations in out-of-distribution (OOD) generalization compared with humans. We investigate this gap through periodicity, one of the basic OOD scenarios. Periodicity captures invariance amid variation. Periodicity generalization represents a model's ability to extract periodic patterns from training data and generalize to OOD scenarios. We introduce a unified interpretation of periodicity from the perspective of abstract algebra and reasoning, including both single and composite periodicity, to explain why Transformers struggle to generalize periodicity. Then we construct Coper about composite periodicity, a controllable generative benchmark with two OOD settings, Hollow and Extrapolation. Experiments reveal that periodicity generalization in Transformers is limited, where models can memorize periodic data during training, but cannot generalize to unseen composite periodicity. We release the source code to support future research.
@article{arxiv.2601.22690,
title = {Do Transformers Have the Ability for Periodicity Generalization?},
author = {Huanyu Liu and Ge Li and Yihong Dong and Sihan Wu and Peixu Wang and Sihao Cheng and Taozhi Chen and Kechi Zhang and Hao Zhu and Tongxuan Liu},
journal= {arXiv preprint arXiv:2601.22690},
year = {2026}
}