Zero-Shot Reinforcement Learning via Function Encoders
Abstract
Although reinforcement learning (RL) can solve many challenging sequential decision making problems, achieving zero-shot transfer across related tasks remains a challenge. The difficulty lies in finding a good representation for the current task so that the agent understands how it relates to previously seen tasks. To achieve zero-shot transfer, we introduce the function encoder, a representation learning algorithm which represents a function as a weighted combination of learned, non-linear basis functions. By using a function encoder to represent the reward function or the transition function, the agent has information on how the current task relates to previously seen tasks via a coherent vector representation. Thus, the agent is able to achieve transfer between related tasks at run time with no additional training. We demonstrate state-of-the-art data efficiency, asymptotic performance, and training stability in three RL fields by augmenting basic RL algorithms with a function encoder task representation.
Cite
@article{arxiv.2401.17173,
title = {Zero-Shot Reinforcement Learning via Function Encoders},
author = {Tyler Ingebrand and Amy Zhang and Ufuk Topcu},
journal= {arXiv preprint arXiv:2401.17173},
year = {2025}
}
Comments
A critical issue was found in the multi-agent experiments published in version 2. We rerun the multi-agent experiments on a more challenging, partially observable Markov game