Wormhole MAML: Meta-Learning in Glued Parameter Space
Machine Learning
2023-01-02 v1 Artificial Intelligence
Abstract
In this paper, we introduce a novel variation of model-agnostic meta-learning, where an extra multiplicative parameter is introduced in the inner-loop adaptation. Our variation creates a shortcut in the parameter space for the inner-loop adaptation and increases model expressivity in a highly controllable manner. We show both theoretically and numerically that our variation alleviates the problem of conflicting gradients and improves training dynamics. We conduct experiments on 3 distinctive problems, including a toy classification problem for threshold comparison, a regression problem for wavelet transform, and a classification problem on MNIST. We also discuss ways to generalize our method to a broader class of problems.
Cite
@article{arxiv.2212.14094,
title = {Wormhole MAML: Meta-Learning in Glued Parameter Space},
author = {Chih-Jung Tracy Chang and Yuan Gao and Beicheng Lou},
journal= {arXiv preprint arXiv:2212.14094},
year = {2023}
}