Distributional Monte Carlo Tree Search for Risk-Aware and Multi-Objective Reinforcement Learning

Conor F. Hayes, Mathieu Reymond, Diederik M. Roijers, Enda Howley, Patrick Mannion

Research output: Chapter in Book/Report/Conference proceedingConference paper

3 Citations (Scopus)

Abstract

In many risk-aware and multi-objective reinforcement learning settings, the utility of the user is derived from the single execution of a policy. In these settings, making decisions based on the average future returns is not suitable. For example, in a medical setting a patient may only have one opportunity to treat their illness. When making a decision, just the expected return -- known in reinforcement learning as the value -- cannot account for the potential range of adverse or positive outcomes a decision may have. Our key insight is that we should use the distribution over expected future returns differently to represent the critical information that the agent requires at decision time. In this paper, we propose Distributional Monte Carlo Tree Search, an algorithm that learns a posterior distribution over the utility of the different possible returns attainable from individual policy executions, resulting in good policies for both risk-aware and multi-objective settings. Moreover, our algorithm outperforms the state-of-the-art in multi-objective reinforcement learning for the expected utility of the returns.
Original languageEnglish
Title of host publicationThe 20th International Conference on Autonomous Agents and Multiagent Systems
PublisherIFAAMAS
Pages1518-1520
Number of pages3
ISBN (Electronic)9781713832621
Publication statusPublished - 3 May 2021
EventThe 20th International Conference on Autonomous Agents and Multiagent Systems - Virtual
Duration: 3 May 20217 May 2021
https://aamas2021.soton.ac.uk/

Publication series

NameProceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
Volume3
ISSN (Print)1548-8403
ISSN (Electronic)1558-2914

Conference

ConferenceThe 20th International Conference on Autonomous Agents and Multiagent Systems
Abbreviated titleAAMAS 2021
Period3/05/217/05/21
Internet address

Fingerprint

Dive into the research topics of 'Distributional Monte Carlo Tree Search for Risk-Aware and Multi-Objective Reinforcement Learning'. Together they form a unique fingerprint.

Cite this