Continual reinforcement learning (CRL) empowers RL agents with the ability to learn a sequence of tasks, accumulating knowledge learned in the past and using the knowledge for problemsolving or future task learning. However, existing methods often focus on transferring fine-grained knowledge across similar tasks, which neglects the multi-granularity structure of human cognitive control, resulting in insufficient knowledge transfer across diverse tasks. To enhance coarse-grained knowledge transfer, we propose a novel framework called MT-Core (as shorthand for Multi-granularity knowledge Transfer for Continual reinforcement learning). MT-Core has a key characteristic of multi-granularity policy learning: 1) a coarsegrained policy formulation for utilizing the powerful reasoning ability of the large language model (LLM) to set goals, and 2) a fine-grained policy learning through RL which is oriented by the goals. We also construct a new policy library (knowledge base) to store policies that can be retrieved for multi-granularity knowledge transfer. Experimental results demonstrate the superiority of the proposed MT-Core in handling diverse CRL tasks versus popular baselines.
@article{arxiv.2401.15098,
title = {Multi-granularity Knowledge Transfer for Continual Reinforcement Learning},
author = {Chaofan Pan and Lingfei Ren and Yihui Feng and Linbo Xiong and Wei Wei and Yonghao Li and Xin Yang},
journal= {arXiv preprint arXiv:2401.15098},
year = {2025}
}
Comments
the 34th International Joint Conference on Artificial Intelligence (IJCAI 2025)