Robust POMDPs extend classical POMDPs to incorporate model uncertainty using so-called uncertainty sets on the transition and observation functions, effectively defining ranges of probabilities. Policies for robust POMDPs must be (1) memory-based to account for partial observability and (2) robust against model uncertainty to account for the worst-case probability instances from the uncertainty sets. To compute such robust memory-based policies, we propose the pessimistic iterative planning (PIP) framework, which alternates between (1) selecting pessimistic POMDPs via worst-case probability instances from the uncertainty sets, and (2) computing finite-state controllers (FSCs) for these pessimistic POMDPs. Within PIP, we propose the RFSCNET algorithm, which optimizes a recurrent neural network to compute the FSCs. The empirical evaluation shows that RFSCNET can compute better-performing robust policies than several baselines and a state-of-the-art robust POMDP solver.

Citation

  Galesloot, M., Suilen, M., Simão, T. D., Carr, S., Spaan, M. T. J., Topcu, U., & Jansen, N. (2025). Pessimistic Iterative Planning with RNNs for Robust POMDPs. ECAI, 4823–4831.

@inproceedings{Galesloot2025pessimistic,
  author = {Galesloot, Maris and Suilen, Marnix and Sim{\~a}o, Thiago D. and Carr, Steven and Spaan, Matthijs T. J. and Topcu, Ufuk and Jansen, Nils},
  title = {Pessimistic Iterative Planning with {RNN}s for Robust {POMDP}s},
  booktitle = {ECAI},
  year = {2025},
  pages = {4823--4831}
}