Multivariate Normal Distribution Based Multi-Armed Bandit Pareto Algorithm

Saba Yahyaa; Madalina Drugan; Bernard Manderick

Abstract

In the stochastic multivariate multi-armed bandit,
arms generate a vector of stochastic normal rewards, one per objective, instead of a single scalar reward.
As a result, there is not only one optimal arm,
but there is a set of optimal arms (Pareto front)
using
Pareto dominance relation.
The goal of an agent is to trade-off between exploration and exploitation.
Exploration means finding the Pareto front and exploitation means
selecting fairly or evenly the optimal arms.
We propose annealing-Pareto algorithm that
trades-off between exploration and exploitation
by using a decaying parameter $\epsilon_{t}$ in combination with
Pareto dominance relation.
We compare experimentally Pareto-KG, Pareto-UCB1 and annealing-Pareto
on multi-objective normal distributions and we conclude that the annealing-Pareto is the best performing algorithm.

Original language	English
Title of host publication	the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track
Publication status	Published - 2014

Publication series

Name	the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track

Keywords

Multi armed bandit problem
multi objective optimization
annealing algorithm
exploration/exploitation

Cite this

Yahyaa, S., Drugan, M., & Manderick, B. (2014). Multivariate Normal Distribution Based Multi-Armed Bandit Pareto Algorithm. In the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track (the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track).

Yahyaa, Saba ; Drugan, Madalina ; Manderick, Bernard. / Multivariate Normal Distribution Based Multi-Armed Bandit Pareto Algorithm. the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track. 2014. (the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track).

@inbook{3dfc08989cf24c6889ae457cb429db8b,

title = "Multivariate Normal Distribution Based Multi-Armed Bandit Pareto Algorithm",

abstract = "In the stochastic multivariate multi-armed bandit, arms generate a vector of stochastic normal rewards, one per objective, instead of a single scalar reward. As a result, there is not only one optimal arm, but there is a set of optimal arms (Pareto front) using Pareto dominance relation. The goal of an agent is to trade-off between exploration and exploitation. Exploration means finding the Pareto front and exploitation means selecting fairly or evenly the optimal arms. We propose annealing-Pareto algorithm that trades-off between exploration and exploitation by using a decaying parameter $\epsilon_{t}$ in combination with Pareto dominance relation. We compare experimentally Pareto-KG, Pareto-UCB1 and annealing-Pareto on multi-objective normal distributions and we conclude that the annealing-Pareto is the best performing algorithm.",

keywords = "Multi armed bandit problem, multi objective optimization, annealing algorithm, exploration/exploitation",

author = "Saba Yahyaa and Madalina Drugan and Bernard Manderick",

year = "2014",

language = "English",

series = "the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track",

booktitle = "the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track",

}

Yahyaa, S, Drugan, M & Manderick, B 2014, Multivariate Normal Distribution Based Multi-Armed Bandit Pareto Algorithm. in the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track. the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track.

Multivariate Normal Distribution Based Multi-Armed Bandit Pareto Algorithm. / Yahyaa, Saba; Drugan, Madalina; Manderick, Bernard.

the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track. 2014. (the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track).

Research output: Chapter in Book/Report/Conference proceeding › Meeting abstract (Book) › Research

TY - CHAP

T1 - Multivariate Normal Distribution Based Multi-Armed Bandit Pareto Algorithm

AU - Yahyaa, Saba

AU - Drugan, Madalina

AU - Manderick, Bernard

PY - 2014

Y1 - 2014

N2 - In the stochastic multivariate multi-armed bandit, arms generate a vector of stochastic normal rewards, one per objective, instead of a single scalar reward. As a result, there is not only one optimal arm, but there is a set of optimal arms (Pareto front) using Pareto dominance relation. The goal of an agent is to trade-off between exploration and exploitation. Exploration means finding the Pareto front and exploitation means selecting fairly or evenly the optimal arms. We propose annealing-Pareto algorithm that trades-off between exploration and exploitation by using a decaying parameter $\epsilon_{t}$ in combination with Pareto dominance relation. We compare experimentally Pareto-KG, Pareto-UCB1 and annealing-Pareto on multi-objective normal distributions and we conclude that the annealing-Pareto is the best performing algorithm.

AB - In the stochastic multivariate multi-armed bandit, arms generate a vector of stochastic normal rewards, one per objective, instead of a single scalar reward. As a result, there is not only one optimal arm, but there is a set of optimal arms (Pareto front) using Pareto dominance relation. The goal of an agent is to trade-off between exploration and exploitation. Exploration means finding the Pareto front and exploitation means selecting fairly or evenly the optimal arms. We propose annealing-Pareto algorithm that trades-off between exploration and exploitation by using a decaying parameter $\epsilon_{t}$ in combination with Pareto dominance relation. We compare experimentally Pareto-KG, Pareto-UCB1 and annealing-Pareto on multi-objective normal distributions and we conclude that the annealing-Pareto is the best performing algorithm.

KW - Multi armed bandit problem

KW - multi objective optimization

KW - annealing algorithm

KW - exploration/exploitation

M3 - Meeting abstract (Book)

T3 - the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track

BT - the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track

ER -

Yahyaa S, Drugan M, Manderick B. Multivariate Normal Distribution Based Multi-Armed Bandit Pareto Algorithm. In the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track. 2014. (the 7th European conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (EMCL/PKDD), PhD track).

Multivariate Normal Distribution Based Multi-Armed Bandit Pareto Algorithm

Abstract

Publication series

Keywords

Fingerprint

Cite this