Projects per year
Abstract
This paper introduces a novel approach to visual dialogue that is based on neuro-symbolic procedural semantics. The approach builds further on earlier work on procedural semantics for visual question answering and expands it on the one hand with neuro-symbolic reasoning operations, and on the other hand with mechanisms that handle the challenges that are inherent to dialogue, in particular the incremental nature of the information that is conveyed. Concretely, we introduce (i) the use of a conversation memory as a data structure that explicitly and incrementally represents the information that is expressed during the subsequent turns of a dialogue, and (ii) the design of a neuro-symbolic procedural semantic representation that is grounded in both visual input and the conversation memory. We validate the methodology using the reasoning- intensive MNIST Dialog and CLEVR-Dialog benchmark challenges and achieve a question-level accuracy of 99.8% and 99.2% respectively. The methodology presented in this paper responds to the growing interest in the field of artificial intelligence in solving tasks that involve both low-level perception and high-level reasoning using a combination of neural and symbolic techniques.
Original language | English |
---|---|
Title of host publication | Proceedings of the 26th European Conference on Artificial Intelligence |
Subtitle of host publication | ECAI 2023 |
Publisher | IOS Press |
Pages | 2419-2426 |
Number of pages | 8 |
Volume | 372 |
ISBN (Electronic) | 978-1-64368-437-6 |
ISBN (Print) | 978-1-64368-436-9 |
DOIs | |
Publication status | Published - 2023 |
Event | 26th European Conference on Artificial Intelligence - ECAI 2023 - Kraków, Poland Duration: 30 Sept 2023 → 4 Oct 2023 Conference number: 26 https://ecai2023.eu/ |
Publication series
Name | Frontiers in Artificial Intelligence and Applications |
---|
Conference
Conference | 26th European Conference on Artificial Intelligence - ECAI 2023 |
---|---|
Abbreviated title | ECAI 2023 |
Country/Territory | Poland |
City | Kraków |
Period | 30/09/23 → 4/10/23 |
Internet address |
Bibliographical note
Funding Information:The research reported on in this paper received funding from the EU’s H2020 RIA programme under grant agreement no. 951846 (MUHAI), from the Research Foundation Flanders (FWO) through a postdoctoral grant awarded to PVE (grant no. 76929) and from the Flemish Government under the ‘Flanders AI Research Program’.
Publisher Copyright:
© 2023 The Authors.
Fingerprint
Dive into the research topics of 'Neuro-Symbolic Procedural Semantics for Reasoning-Intensive Visual Dialogue Tasks'. Together they form a unique fingerprint.-
VLAAI1: Flanders Artificial Intelligence Research program (FAIR) – second cycle
1/01/24 → 31/12/28
Project: Applied
-
EU629: Meaning and Understanding in Human-centric AI
Beuls, K., Nowe, A. & Van Eecke, P.
1/10/20 → 31/03/25
Project: Fundamental
Datasets
-
babel
Van Eecke, P. (Creator), Verheyen, L. (Creator) & Botoko Ekila, J. (Creator), VUB, 2025
https://gitlab.ai.vub.ac.be/ehai/babel
Dataset