Local Advantage Networks for Cooperative Multi-Agent Reinforcement Learning

Research output: Chapter in Book/Report/Conference proceedingConference paper

1 Citation (Scopus)

Abstract

Multi-agent reinforcement learning (MARL) enables us to create adaptive agents in challenging environments, even when the agents have limited observation. Modern MARL methods have focused on finding factorized value functions. While successful, the resulting methods have convoluted network structures. We take a radically different approach and build on the structure of independent Q-learners. Our algorithm LAN leverages a dueling architecture to represent decentralized policies as separate individual advantage functions w.r.t.\ a centralized critic that is cast aside after training. The critic works as a stabilizer that coordinates the learning and to formulate DQN targets. This enables LAN to keep the number of parameters of its centralized network independent in the number of agents, without imposing additional constraints like monotonic value functions.
When evaluated on the SMAC, LAN shows SOTA performance overall and scores more than 80\% wins in two super-hard maps where even QPLEX does not obtain almost any wins. Moreover, when the number of agents becomes large, LAN uses significantly fewer parameters than QPLEX or even QMIX. We thus show that LAN's structure forms a key improvement that helps MARL methods remain scalable.
Original languageEnglish
Title of host publicationInternational Conference on Autonomous Agents and Multiagent Systems, AAMAS 2022
Subtitle of host publicationExtended Abstract
PublisherIFAAMAS
Pages1524-1526
Number of pages3
ISBN (Electronic)9781713854333
Publication statusPublished - 9 May 2022
Event21st International Conference on Autonomous Agents and Multi-agent System -
Duration: 9 May 202213 May 2022
Conference number: 21
https://aamas2022-conference.auckland.ac.nz

Publication series

NameProceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
Volume3
ISSN (Print)1548-8403
ISSN (Electronic)1558-2914

Conference

Conference21st International Conference on Autonomous Agents and Multi-agent System
Abbreviated titleAAMAS
Period9/05/2213/05/22
Internet address

Bibliographical note

Funding Information:
Raphaël Avalos was supported by the FWO (grant 11F5721N). This research was supported by the Flemish Government under the “Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen” program.

Publisher Copyright:
© 2022 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.

Copyright:
Copyright 2022 Elsevier B.V., All rights reserved.

Fingerprint

Dive into the research topics of 'Local Advantage Networks for Cooperative Multi-Agent Reinforcement Learning'. Together they form a unique fingerprint.

Cite this