Multi-Objectivization of Reinforcement Learning Problems by Reward Shaping

Tim Brys, Anna Harutyunyan, Peter Vrancx, Daniel Kudenko, Matthew Taylor, Ann Nowe

Onderzoeksoutput: Conference paper

43 Citaten (Scopus)

Samenvatting

Multi-objectivization is the process of transforming a single objective problem into a multi-objective problem. Research in evolutionary optimization has demonstrated that the addition of objectives that are correlated with the original objective can make the resulting problem easier to solve compared to the original single-objective problem. In this paper we investigate the multi-objectivization of reinforcement learning problems. We propose a novel method for the multi-objectivization of Markov Decision problems through the use of multiple reward shaping functions. Reward shaping is a technique to speed up reinforcement learning by including additional heuristic knowledge in the reward signal. The resulting composite reward signal is expected to be more informative during learning, leading the learner to identify good actions more quickly. Good reward shaping functions are by definition correlated with the target value function for the base reward signal, and we show in this paper that adding several correlated signals can help to solve the basic single objective problem faster and better. We prove that the total ordering of solutions, and by consequence the optimality of solutions, is preserved in this process, and empirically demonstrate the usefulness of this approach on two reinforcement learning tasks: a pathfinding problem and the Mario domain.
Originele taal-2English
TitelInternational Joint Conference on Neural Networks
Pagina's2315-2322
ISBN van elektronische versie978-1-4799-1484-5
StatusPublished - 2014
Evenement2014 International Joint Conference on Neural Networks (IJCNN) - Beijing, China
Duur: 6 jul 201411 jul 2014

Publicatie series

NaamInternational Joint Conference on Neural Networks

Conference

Conference2014 International Joint Conference on Neural Networks (IJCNN)
Land/RegioChina
StadBeijing
Periode6/07/1411/07/14

Vingerafdruk

Duik in de onderzoeksthema's van 'Multi-Objectivization of Reinforcement Learning Problems by Reward Shaping'. Samen vormen ze een unieke vingerafdruk.

Citeer dit