Projects per year
In interactive multi-objective reinforcement learning (MORL), an agent has to simultaneously learn about the environment and the preferences of the user, in order to quickly zoom in on those decisions that are likely to be preferred by the user. In this paper we study interactive MORL in the context of multi-objective multi-armed bandits. Contrary to earlier approaches to interactive MORL that force the utility of the user to be expressed as a weighted sum of the values for each objective, we do not make such stringent a priori assumptions. Specifically, we not only allow non-linear preferences, but also obviate the need to specify the exact model class in the utility function must fall. To achieve this, we propose a new approach called Gaussian-process Utility Thompson Sampling (GUTS). GUTS employs parameterless Bayesian learning to allow any type of utility function, exploits monotonicity information, and limits the number of queries posed to the user by ensuring that questions are statistically significant. We show empirically that GUTS can learn non-linear preferences, and that the regret and number of queries posed to the user are highly sub-linear in the number of arm pulls. (A preliminary version of this work was presented at the ALA workshop in 2018 ).
|Title of host publication||ECML-PKDD 2020: Proceedings of the 2020 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases|
|Editors||Frank Hutter, Kristian Kersting, Jefrey Lijffijt, Isabel Valera|
|Number of pages||16|
|Publication status||Published - 2021|
|Event||ECML PKDD: Joint European Conference on Machine Learning and Knowledge Discovery in Databases|
- Ghent, Belgium
Duration: 14 Sep 2020 → 18 Sep 2020
|Conference||ECML PKDD: Joint European Conference on Machine Learning and Knowledge Discovery in Databases|
|Period||14/09/20 → 18/09/20|
FingerprintDive into the research topics of 'Interactive Multi-Objective Reinforcement Learning in Multi-Armed Bandits with Gaussian Process Utility Models'. Together they form a unique fingerprint.
- 1 Active
1/07/19 → 31/12/24