Pessimistic Iterative Planning with RNNs for Robust POMDPs
Abstract
Robust POMDPs extend classical POMDPs to incorporate model uncertainty using so-called uncertainty sets on the transition and observation functions, effectively defining ranges of probabilities. Policies for robust POMDPs must be (1) memory-based to account for partial observability and (2) robust against model uncertainty to account for the worst-case probability instances from the uncertainty sets. To compute such robust memory-based policies, we propose the pessimistic iterative planning (PIP) framework, which alternates between (1) selecting pessimistic POMDPs via worst-case probability instances from the uncertainty sets, and (2) computing finite-state controllers (FSCs) for these pessimistic POMDPs. Within PIP, we propose the rFSCNet algorithm, which optimizes a recurrent neural network to compute the FSCs. The empirical evaluation shows that rFSCNet can compute better-performing robust policies than several baselines and a state-of-the-art robust POMDP solver.
Cite
@article{arxiv.2408.08770,
title = {Pessimistic Iterative Planning with RNNs for Robust POMDPs},
author = {Maris F. L. Galesloot and Marnix Suilen and Thiago D. Simão and Steven Carr and Matthijs T. J. Spaan and Ufuk Topcu and Nils Jansen},
journal= {arXiv preprint arXiv:2408.08770},
year = {2025}
}
Comments
Accepted for presentation at ECAI 2025