Projects per year
Abstract
Multi-agent reinforcement learning (MARL) enables us to create adaptive agents in challenging environments, even when the agents have limited observation. Modern MARL methods have focused on finding factorized value functions. While successful, the resulting methods have convoluted network structures. We take a radically different approach and build on the structure of independent Q-learners. Our algorithm LAN leverages a dueling architecture to represent decentralized policies as separate individual advantage functions w.r.t.\ a centralized critic that is cast aside after training. The critic works as a stabilizer that coordinates the learning and to formulate DQN targets. This enables LAN to keep the number of parameters of its centralized network independent in the number of agents, without imposing additional constraints like monotonic value functions.
When evaluated on the SMAC, LAN shows SOTA performance overall and scores more than 80\% wins in two super-hard maps where even QPLEX does not obtain almost any wins. Moreover, when the number of agents becomes large, LAN uses significantly fewer parameters than QPLEX or even QMIX. We thus show that LAN's structure forms a key improvement that helps MARL methods remain scalable.
When evaluated on the SMAC, LAN shows SOTA performance overall and scores more than 80\% wins in two super-hard maps where even QPLEX does not obtain almost any wins. Moreover, when the number of agents becomes large, LAN uses significantly fewer parameters than QPLEX or even QMIX. We thus show that LAN's structure forms a key improvement that helps MARL methods remain scalable.
Original language | English |
---|---|
Title of host publication | International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2022 |
Subtitle of host publication | Extended Abstract |
Publisher | IFAAMAS |
Pages | 1524-1526 |
Number of pages | 3 |
ISBN (Electronic) | 9781713854333 |
Publication status | Published - 9 May 2022 |
Event | 21st International Conference on Autonomous Agents and Multi-agent System - Duration: 9 May 2022 → 13 May 2022 Conference number: 21 https://aamas2022-conference.auckland.ac.nz |
Publication series
Name | Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS |
---|---|
Volume | 3 |
ISSN (Print) | 1548-8403 |
ISSN (Electronic) | 1558-2914 |
Conference
Conference | 21st International Conference on Autonomous Agents and Multi-agent System |
---|---|
Abbreviated title | AAMAS |
Period | 9/05/22 → 13/05/22 |
Internet address |
Bibliographical note
Funding Information:Raphaël Avalos was supported by the FWO (grant 11F5721N). This research was supported by the Flemish Government under the “Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen” program.
Publisher Copyright:
© 2022 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.
Copyright:
Copyright 2022 Elsevier B.V., All rights reserved.
Fingerprint
Dive into the research topics of 'Local Advantage Networks for Cooperative Multi-Agent Reinforcement Learning'. Together they form a unique fingerprint.Projects
- 1 Active
-
VLAAI1: Subsidie: Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen
1/07/19 → 31/12/23
Project: Applied