Scalable Meta-Learning with Gaussian Processes
Abstract
Meta-learning is a powerful approach that exploits historical data to quickly solve new tasks from the same distribution. In the low-data regime, methods based on the closed-form posterior of Gaussian processes (GP) together with Bayesian optimization have achieved high performance. However, these methods are either computationally expensive or introduce assumptions that hinder a principled propagation of uncertainty between task models. This may disrupt the balance between exploration and exploitation during optimization. In this paper, we develop ScaML-GP, a modular GP model for meta-learning that is scalable in the number of tasks. Our core contribution is a carefully designed multi-task kernel that enables hierarchical training and task scalability. Conditioning ScaML-GP on the meta-data exposes its modular nature yielding a test-task prior that combines the posteriors of meta-task GPs. In synthetic and real-world meta-learning experiments, we demonstrate that ScaML-GP can learn efficiently both with few and many meta-tasks.
Cite
@article{arxiv.2312.00742,
title = {Scalable Meta-Learning with Gaussian Processes},
author = {Petru Tighineanu and Lukas Grossberger and Paul Baireuther and Kathrin Skubch and Stefan Falkner and Julia Vinogradska and Felix Berkenkamp},
journal= {arXiv preprint arXiv:2312.00742},
year = {2023}
}