Cooperative Prioritized Sweeping

Eugenio Bargiacchi; Timothy Verstraeten; Diederik M. Roijers

doi:10.5555/3463952.3463977

Cooperative Prioritized Sweeping

Eugenio Bargiacchi, Timothy Verstraeten, Diederik M. Roijers

Research output: Chapter in Book/Report/Conference proceeding › Conference paper

1 Citation (Scopus)

22 Downloads (Pure)

Abstract

We present a novel model-based algorithm, Cooperative Prioritized Sweeping, for sample-efficient learning in large multi-agent Markov decision processes. Our approach leverages domain knowledge about the structure of the problem in the form of a dynamic decision network. Using this information, our method learns a model of the environment to determine which state-action pairs are the most likely in need to be updated, significantly increasing learning speed. Batch updates can then be performed which efficiently back-propagate knowledge throughout the value function. Our method outperforms the state-of-the-art sparse cooperative Q-learning and QMIX algorithms, both on the well-known SysAdmin benchmark, randomized environments and a fully-observable variation of the well-known firefighter benchmark from Dec-POMDP literature.

Original language	English
Title of host publication	Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2021
Publisher	IFAAMAS
Pages	160-168
Number of pages	9
ISBN (Electronic)	9781713832621
DOIs	https://doi.org/10.5555/3463952.3463977
Publication status	Published - 2021
Event	The 20th International Conference on Autonomous Agents and Multiagent Systems - Virtual Duration: 3 May 2021 → 7 May 2021 https://aamas2021.soton.ac.uk/

Publication series

Name	Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
Volume	1
ISSN (Print)	1548-8403
ISSN (Electronic)	1558-2914

Conference

Conference	The 20th International Conference on Autonomous Agents and Multiagent Systems
Abbreviated title	AAMAS 2021
Period	3/05/21 → 7/05/21
Internet address	https://aamas2021.soton.ac.uk/

Access to Document

10.5555/3463952.3463977Licence: Unspecified

p160Final published version, 1.13 MBLicence: Unspecified

https://www.ifaamas.org/Proceedings/aamas2021/pdfs/p160.pdfLicence: Unspecified
Link to publication in Scopus

1 Active
2 Finished

VLAAI1: Subsidie: Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen
Nowe, A. & Vanderborght, B.
1/07/19 → 31/12/23
Project: Applied
FWOSB87: Controlling large scale multi-agent environments with model-based reinforcement learning
Nowe, A. & Bargiacchi, E.
1/11/19 → 31/10/21
Project: Fundamental
FWOSB27: Robust Fleet-Wide Reinforcement Learning
Nowe, A., Verstraeten, T. & Helsen, J.
1/01/17 → 31/12/20
Project: Fundamental

Cite this

Bargiacchi, E., Verstraeten, T., & Roijers, D. M. (2021). Cooperative Prioritized Sweeping. In Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2021 (pp. 160-168). (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS; Vol. 1). IFAAMAS. https://doi.org/10.5555/3463952.3463977

@inproceedings{3384dea92eff43eeb006f91bc90b43f5,

title = "Cooperative Prioritized Sweeping",

abstract = "We present a novel model-based algorithm, Cooperative Prioritized Sweeping, for sample-efficient learning in large multi-agent Markov decision processes. Our approach leverages domain knowledge about the structure of the problem in the form of a dynamic decision network. Using this information, our method learns a model of the environment to determine which state-action pairs are the most likely in need to be updated, significantly increasing learning speed. Batch updates can then be performed which efficiently back-propagate knowledge throughout the value function. Our method outperforms the state-of-the-art sparse cooperative Q-learning and QMIX algorithms, both on the well-known SysAdmin benchmark, randomized environments and a fully-observable variation of the well-known firefighter benchmark from Dec-POMDP literature.",

author = "Eugenio Bargiacchi and Timothy Verstraeten and Roijers, {Diederik M.}",

year = "2021",

doi = "10.5555/3463952.3463977",

language = "English",

series = "Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS",

publisher = "IFAAMAS",

pages = "160--168",

booktitle = "Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2021",

}

Bargiacchi, E , Verstraeten, T & Roijers, DM 2021, Cooperative Prioritized Sweeping. in Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2021. Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 1, IFAAMAS, pp. 160-168, The 20th International Conference on Autonomous Agents and Multiagent Systems, 3/05/21. https://doi.org/10.5555/3463952.3463977

Cooperative Prioritized Sweeping. / Bargiacchi, Eugenio ; Verstraeten, Timothy ; Roijers, Diederik M.

Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2021. IFAAMAS, 2021. p. 160-168 (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS; Vol. 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference paper

TY - GEN

T1 - Cooperative Prioritized Sweeping

AU - Bargiacchi, Eugenio

AU - Verstraeten, Timothy

AU - Roijers, Diederik M.

PY - 2021

Y1 - 2021

N2 - We present a novel model-based algorithm, Cooperative Prioritized Sweeping, for sample-efficient learning in large multi-agent Markov decision processes. Our approach leverages domain knowledge about the structure of the problem in the form of a dynamic decision network. Using this information, our method learns a model of the environment to determine which state-action pairs are the most likely in need to be updated, significantly increasing learning speed. Batch updates can then be performed which efficiently back-propagate knowledge throughout the value function. Our method outperforms the state-of-the-art sparse cooperative Q-learning and QMIX algorithms, both on the well-known SysAdmin benchmark, randomized environments and a fully-observable variation of the well-known firefighter benchmark from Dec-POMDP literature.

AB - We present a novel model-based algorithm, Cooperative Prioritized Sweeping, for sample-efficient learning in large multi-agent Markov decision processes. Our approach leverages domain knowledge about the structure of the problem in the form of a dynamic decision network. Using this information, our method learns a model of the environment to determine which state-action pairs are the most likely in need to be updated, significantly increasing learning speed. Batch updates can then be performed which efficiently back-propagate knowledge throughout the value function. Our method outperforms the state-of-the-art sparse cooperative Q-learning and QMIX algorithms, both on the well-known SysAdmin benchmark, randomized environments and a fully-observable variation of the well-known firefighter benchmark from Dec-POMDP literature.

UR - http://www.scopus.com/inward/record.url?scp=85112236405&partnerID=8YFLogxK

U2 - 10.5555/3463952.3463977

DO - 10.5555/3463952.3463977

M3 - Conference paper

T3 - Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS

SP - 160

EP - 168

BT - Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2021

PB - IFAAMAS

ER -

Cooperative Prioritized Sweeping

Abstract

Publication series

Conference

Access to Document

Fingerprint

Projects

VLAAI1: Subsidie: Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen

FWOSB87: Controlling large scale multi-agent environments with model-based reinforcement learning

FWOSB27: Robust Fleet-Wide Reinforcement Learning

Cite this