Projects per year
We consider the challenge of policy simplification and verification in the context of policies learned through reinforcement learning (RL) in continuous environments. In well-behaved settings, RL algorithms have convergence guarantees in the limit. While these guarantees are valuable, they are insufficient for safety-critical applications. Furthermore, they are lost when applying advanced techniques such as deep-RL. To recover guarantees when applying advanced RL algorithms to more complex environments with (i) reachability, (ii) safety-constrained reachability, or (iii) discounted-reward objectives, we build upon the DeepMDP framework to derive new bisimulation bounds between the unknown environment and a learned discrete latent model of it. Our bisimulation bounds enable the application of formal methods for Markov decision processes. Finally, we show how one can use a policy obtained via state-of-the-art RL to efficiently train a variational autoencoder that yields a discrete latent model with provably approximately correct bisimulation guarantees. Additionally, we obtain a distilled version of the policy for the latent model.
|Publication status||Published - 7 Nov 2022|
|Event||BNAIC/BeNeLearn 2022: Joint International Scientific Conferences on AI and Machine Learning - Lamot Mechelen, Belgium|
Duration: 7 Nov 2022 → 9 Nov 2022
|Abbreviated title||BNAIC/BeNeLearn 2022|
|Period||7/11/22 → 9/11/22|
- Reinforcement Learning
- Formal Methods
- Representation Learning
- 2 Active
iBOF/21/027: DESCARTES - infectious DisEaSe eConomics and Ai with guaRanTEeS
Nowe, A., Hens, N. & Beutels, P.
1/01/21 → 31/12/24
VLAAI1: Subsidie: Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen
1/07/19 → 31/12/23
- 1 Conference paper
Distillation of RL Policies with Formal Guarantees via Variational Abstraction of Markov Decision ProcessesDelgrange, F., Nowé, A. & Pérez, G. A., 28 Jun 2022, Proceedings of the AAAI Conference on Artificial Intelligence: Vol. 36 No. 6: AAAI-22 Technical Tracks 6. First ed. Palo Alto, California USA: AAAI Press, Vol. 36. p. 6497-6505 9 p. (Proceedings of the AAAI Conference on Artificial Intelligence; vol. 36, no. 6).
Research output: Chapter in Book/Report/Conference proceeding › Conference paperOpen AccessFile19 Downloads (Pure)