Computational construction grammar for visual question answering
Bibliographic Data
| ID | 21226765 |
|---|---|
| Authors | Jens Nevens (0000-0001-5413-2955, Vrije Universiteit Brussel, corresponding author), Paul Van Eecke (0000-0001-9153-9092, Vrije Universiteit Brussel), Katrien Beuls (0000-0003-4451-4778, Vrije Universiteit Brussel) |
| Year | 2019 |
| Volume | 5 |
| Issue | 1 |
| Publication date | 2019-01-28 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Linguistics Vanguard (JOURNAL) |
| Journal identifiers | ISSN: 2199-174X • E-ISSN: 2199-174X |
| Publisher | Walter de Gruyter GmbH (PUBLISHER • DE) |
| DOI | 10.1515/lingvan-2018-0070 |
| OpenAlex | W2994718931 |
| Language | EN |
| Citations received | 2 |
| References cited | 21 |
In order to be able to answer a natural language question, a computational system needs three main capabilities. First, the system needs to be able to analyze the question into a structured query, revealing its component parts and how these are combined. Second, it needs to have access to relevant knowledge sources, such as databases, texts or images. Third, it needs to be able to execute the query on these knowledge sources. This paper focuses on the first capability, presenting a novel approach to semantically parsing questions expressed in natural language. The method makes use of a computational construction grammar model for mapping questions onto their executable semantic representations. We demonstrate and evaluate the methodology on the CLEVR visual question answering benchmark task. Our system achieves a 100% accuracy, effectively solving the language understanding part of the benchmark task. Additionally, we demonstrate how this solution can be embedded in a full visual question answering system, in which a question is answered by executing its semantic representation on an image. The main advantages of the approach include (i) its transparent and interpretable properties, (ii) its extensibility, and (iii) the fact that the method does not rely on any annotated training data
Executable · Grammar · Information retrieval · Knowledge representation and reasoning · Linguistics · Natural language · Natural language processing · Parsing · Programming language · Question answering · Advanced Image and Video Retrieval Techniques · Computer Science · Domain Adaptation and Few-Shot Learning · Multimodal Machine Learning Applications · Artificial Intelligence
| Citations per year | 0.67 |
|---|---|
| Citation span | 2023 - 2024 (2) |
| Highly cited | No |