Automatic Annotation Suggestions for Audiovisual Archives
Evaluation Aspects
Bibliographic Data
| ID | 11937672 |
|---|---|
| Authors | Luit Gazendam (Novay, corresponding author), Christian Wartena (0000-0001-5483-1529, Novay), Véronique Malaisé (Vrije Universiteit Amsterdam), Guus Schreiber (Vrije Universiteit Amsterdam), Annemieke de Jong (0000-0001-6707-6585, Netherlands Institute for Sound and Vision), Hennie Brugman (0000-0002-8402-1831, Max Planck Society) |
| Year | 2009 |
| Volume | 34 |
| Issue | 2-3 |
| Pages | 172-188 |
| Publication date | 2009-07-10 |
| Peer Reviewed | Yes |
| Open Access | No |
| Type | ARTICLE |
| Venue | Interdisciplinary Science Reviews (JOURNAL) |
| Journal identifiers | ISSN: 0308-0188 • E-ISSN: 1743-2790 |
| Publisher | Taylor & Francis (PUBLISHER • GB) |
| DOI | 10.1179/174327909x441090 |
| OpenAlex | W2139952202 |
| Language | EN |
| Citations received | 2 |
| References cited | 7 |
In the context of large and ever growing archives, generating annotation suggestions automatically from textual resources related to the documents to be archived is an interesting option in theory. It could save a lot of work in the time consuming and expensive task of manual annotation and it could help cataloguers attain a higher inter-annotator agreement. However, some questions arise in practice: what is the quality of the automatically produced annotations? How do they compare with manual annotations and with the requirements for annotation that were defined in the archive? If different from the manual annotations, are the automatic annotations wrong? In the CHOICE project, partially hosted at the Netherlands Institute for Sound and Vision, the Dutch public archive for audiovisual broadcasts, we automatically generate annotation suggestions for cataloguers. In this paper, we define three types of evaluation of these annotation suggestions: (1) a classic and strict evaluation measure expressing the overlap between automatically generated keywords and the manual annotations, (2) a loosened evaluation measure for which semantically very similar annotations are also considered as relevant matches, and (3) an in-use evaluation of the usefulness of manual versus automatic annotations in the context of serendipitous browsing. During serendipitous browsing, the annotations (manual or automatic) are used to retrieve and visualize semantically related documents
Annotation · Context (archaeology · Information retrieval · Natural language processing · Quality (philosophy · Task (project management · World Wide Web · Computer Science · Music and Audio Processing · Video Analysis and Summarization · Web Data Mining and Analysis · Artificial Intelligence
| Unique citing works | 2 |
|---|---|
| Citations per year | 0,12 |
| Citation span | 2009 - 2010 (2) |
| Citation velocity | historical |
| Highly cited | No |
| Citation types | Neutral: 2 |