Modelling Grounded Language Acquisition in Autonomous Agents through Task-based Communicative Interactions in Situated Environments

Activiteit: Talk at an external academic organisation


We will present an interpretable approach to the modelling of human-like language acquisition and understanding in autonomous agents. In particular, we will discuss the bootstrapping of grounded construction grammars through task-based communicative interactions in situated environments. Two main mechanisms steer this process: intention reading and pattern finding. Intention reading refers to the capacity to hypothesise about the intended meaning of an observed utterance based on the situational context in which it is uttered. Pattern finding refers to the ability to generalise over pairs of observed utterances and their hypothesised meaning, yielding form/meaning mappings of varying degrees of abstraction. These form/meaning mappings, called constructions, capture all linguistic information that a language user needs in order to be able to communicate with other language users, i.e. to operationalise the processes of language comprehension and production in grounded situated environments.

We will start our talk by introducing the construction grammar framework, and in particular the Fluid Construction Grammar (FCG) platform. FCG provides a formalism for representing construction grammars, a processing engine that supports construction-based language comprehension and production, and a set of learning mechanisms for bootstrapping construction grammars through situated communicative interactions between autonomous agents. We will then discuss how the processes of intention reading and pattern finding are computationally operationalised within the FCG framework. Finally, we will provide an overview of applications of this research programme, including in the domains visual question answering, robot instruction and opinion dynamics.
Periode5 mei 2023
Gehouden opUnited States Army Research Laboratory, United States
Mate van erkenningInternational