Local Advantage Networks for Cooperative Multi-Agent Reinforcement Learning

Onderzoeksoutput: Conference paper

1 Citaat (Scopus)

Samenvatting

Multi-agent reinforcement learning (MARL) enables us to create adaptive agents in challenging environments, even when the agents have limited observation. Modern MARL methods have focused on finding factorized value functions. While successful, the resulting methods have convoluted network structures. We take a radically different approach and build on the structure of independent Q-learners. Our algorithm LAN leverages a dueling architecture to represent decentralized policies as separate individual advantage functions w.r.t.\ a centralized critic that is cast aside after training. The critic works as a stabilizer that coordinates the learning and to formulate DQN targets. This enables LAN to keep the number of parameters of its centralized network independent in the number of agents, without imposing additional constraints like monotonic value functions.
When evaluated on the SMAC, LAN shows SOTA performance overall and scores more than 80\% wins in two super-hard maps where even QPLEX does not obtain almost any wins. Moreover, when the number of agents becomes large, LAN uses significantly fewer parameters than QPLEX or even QMIX. We thus show that LAN's structure forms a key improvement that helps MARL methods remain scalable.
Originele taal-2English
TitelInternational Conference on Autonomous Agents and Multiagent Systems, AAMAS 2022
SubtitelExtended Abstract
UitgeverijIFAAMAS
Pagina's1524-1526
Aantal pagina's3
ISBN van elektronische versie9781713854333
StatusPublished - 9 mei 2022
Evenement21st International Conference on Autonomous Agents and Multi-agent System -
Duur: 9 mei 202213 mei 2022
Congresnummer: 21
https://aamas2022-conference.auckland.ac.nz

Publicatie series

NaamProceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
Volume3
ISSN van geprinte versie1548-8403
ISSN van elektronische versie1558-2914

Conference

Conference21st International Conference on Autonomous Agents and Multi-agent System
Verkorte titelAAMAS
Periode9/05/2213/05/22
Internet adres

Bibliografische nota

Funding Information:
Raphaël Avalos was supported by the FWO (grant 11F5721N). This research was supported by the Flemish Government under the “Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen” program.

Publisher Copyright:
© 2022 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.

Copyright:
Copyright 2022 Elsevier B.V., All rights reserved.

Vingerafdruk

Duik in de onderzoeksthema's van 'Local Advantage Networks for Cooperative Multi-Agent Reinforcement Learning'. Samen vormen ze een unieke vingerafdruk.

Citeer dit