Synthesising Reinforcement Learning Policies Through Set-Valued Inductive Rule Learning

Research output: Chapter in Book/Report/Conference proceedingConference paper

2 Citations (Scopus)


Today's advanced Reinforcement Learning algorithms produce black-box policies, that are often difficult to interpret and trust for a person. We introduce a policy distilling algorithm, building on the CN2 rule mining algorithm, that distills the policy into a rule-based decision system. At the core of our approach is the fact that an RL process does not just learn a policy, a mapping from states to actions, but also produces extra meta-information, such as action values indicating the quality of alternative actions. This meta-information can indicate whether more than one action is near-optimal for a certain state. We extend CN2 to make it able to leverage knowledge about equally-good actions to distill the policy into fewer rules, increasing its interpretability by a person. Then, to ensure that the rules explain a valid, non-degenerate policy, we introduce a refinement algorithm that fine-tunes the rules to obtain good performance when executed in the environment. We demonstrate the applicability of our algorithm on the Mario AI benchmark, a complex task that requires modern reinforcement learning algorithms including neural networks. The explanations we produce capture the learned policy in only a few rules, that allow a person to understand what the black-box agent learned. Source code:
Original languageEnglish
Title of host publicationTrustworthy AI - Integrating Learning, Optimization and Reasoning
Subtitle of host publicationFirst International Workshop, TAILOR 2020, Virtual Event, September 4–5, 2020, Revised Selected Papers
EditorsFredrik Heintz, Michela Milano, Barry O'Sullivan
Place of PublicationCham
PublisherSpringer International Publishing
Number of pages17
ISBN (Electronic)978-3-030-73959-1
ISBN (Print)978-3-030-73958-4
Publication statusPublished - 13 Apr 2021
Event1st TAILOR Workshop at ECAI 2020 -
Duration: 4 Sep 20205 Sep 2020

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Workshop1st TAILOR Workshop at ECAI 2020
Internet address

Bibliographical note

Funding Information:
Acknowledgements. This work is supported by the Research Foundation Flanders (FWO) [grant numbers G062819N and 1129319N], the AI Research Program from the Flemish Government (Belgium) and the Francqui Foundation. This work is part of the research program Hybrid Intelligence with project number 024.004.022, which is (partly) financed by the Dutch Ministry of Education, Culture and Science (OCW).

Publisher Copyright:
© 2021, Springer Nature Switzerland AG.

Copyright 2021 Elsevier B.V., All rights reserved.


Dive into the research topics of 'Synthesising Reinforcement Learning Policies Through Set-Valued Inductive Rule Learning'. Together they form a unique fingerprint.

Cite this