Generating Multimodal References

Bibliographic Data

ID11274151
AuthorsIelka van der Sluis (0000-0002-0151-5044, Triangle : Action, Discours, Pensée politique et économique, corresponding author), Emiel Krahmer (0000-0002-6304-7549, Tilburg University)
Year2007
Volume44
Issue3
Pages145-174
Publication date2007-09-01
Peer ReviewedYes
Open AccessNo
TypeARTICLE
VenueDiscourse Processes (JOURNAL)
Journal identifiersISSN: 0163-853X • E-ISSN: 1532-6950
PublisherInforma UK Limited (PUBLISHER • GB)
DOI10.1080/01638530701600755
OpenAlexW2105105514
LanguageEN
Citations received13
References cited10

This paper presents a new computational model for the generation of multimodal referring expressions, based on observations in human communication. The algorithm is an extension of the graph-based algorithm proposed by The Flashlight Model accounts for various types of pointing gestures of different precisions. Based on a notion of effort the algorithm produces referring expressions combining language and pointing gestures. The algorithm is evaluated using two production experiments, with which spontaneous data is gathered on controlled input. The output of the algorithm coincides to a large extent with the utterances of the participants. However, an important difference is that the participants tend to produce overspecified referring expressions while the algorithm generates minimal ones. We briefly discuss ways to generate overspecified multimodal references

Algorithm · Extension (predicate logic · Gesture · Graph · Natural language processing · Programming language · Computer Science · Language, Metaphor, and Cognition · Multi-Agent Systems and Negotiation · Speech and dialogue systems · Artificial Intelligence · Theoretical Computer Science