GmNet: Revisiting Gating Mechanisms From A Frequency View
Abstract
Gating mechanisms have emerged as an effective strategy integrated into model designs beyond recurrent neural networks for addressing long-range dependency problems. In a broad understanding, it provides adaptive control over the information flow while maintaining computational efficiency. However, there is a lack of theoretical analysis on how the gating mechanism works in neural networks. In this paper, inspired by the \textit{convolution theorem}, we systematically explore the effect of gating mechanisms on the training dynamics of neural networks from a frequency perspective. We investigate the interact between the element-wise product and activation functions in managing the responses to different frequency components. Leveraging these insights, we propose a Gating Mechanism Network (GmNet), a lightweight model designed to efficiently utilize the information of various frequency components. It minimizes the low-frequency bias present in existing lightweight models. GmNet achieves impressive performance in terms of both effectiveness and efficiency in the image classification task.
Cite
@article{arxiv.2503.22841,
title = {GmNet: Revisiting Gating Mechanisms From A Frequency View},
author = {Yifan Wang and Xu Ma and Yitian Zhang and Zhongruo Wang and Sung-Cheol Kim and Vahid Mirjalili and Vidya Renganathan and Yun Fu},
journal= {arXiv preprint arXiv:2503.22841},
year = {2026}
}