Interactive Multi-Objective Reinforcement Learning in Multi-Armed Bandits with Gaussian Process Utility Models

Onderzoeksoutput: Conference paper

1 Citaat (Scopus)
107 Downloads (Pure)

Samenvatting

In interactive multi-objective reinforcement learning (MORL), an agent has to simultaneously learn about the environment and the preferences of the user, in order to quickly zoom in on those decisions that are likely to be preferred by the user. In this paper we study interactive MORL in the context of multi-objective multi-armed bandits. Contrary to earlier approaches to interactive MORL that force the utility of the user to be expressed as a weighted sum of the values for each objective, we do not make such stringent a priori assumptions. Specifically, we not only allow non-linear preferences, but also obviate the need to specify the exact model class in the utility function must fall. To achieve this, we propose a new approach called Gaussian-process Utility Thompson Sampling (GUTS). GUTS employs parameterless Bayesian learning to allow any type of utility function, exploits monotonicity information, and limits the number of queries posed to the user by ensuring that questions are statistically significant. We show empirically that GUTS can learn non-linear preferences, and that the regret and number of queries posed to the user are highly sub-linear in the number of arm pulls. (A preliminary version of this work was presented at the ALA workshop in 2018 [20]).

Originele taal-2English
TitelECML-PKDD 2020: Proceedings of the 2020 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases
RedacteurenFrank Hutter, Kristian Kersting, Jefrey Lijffijt, Isabel Valera
UitgeverijSpringer
Aantal pagina's16
ISBN van geprinte versie978-3-030-67664-3
DOI's
StatusPublished - 2021
EvenementECML PKDD: Joint European Conference on Machine Learning and Knowledge Discovery in Databases
- Ghent, Belgium
Duur: 14 sep 202018 sep 2020

Conference

ConferenceECML PKDD: Joint European Conference on Machine Learning and Knowledge Discovery in Databases
LandBelgium
StadGhent
Periode14/09/2018/09/20

Vingerafdruk

Duik in de onderzoeksthema's van 'Interactive Multi-Objective Reinforcement Learning in Multi-Armed Bandits with Gaussian Process Utility Models'. Samen vormen ze een unieke vingerafdruk.

Citeer dit