Anja Belz
Biographic Data
| ID | 1485728 |
|---|---|
| NAME | Anja Belz |
| GIVEN NAMES | Anja |
| FAMILY NAME | Belz |
| SIGNATURE | BELZ A |
| AFFILIATIONS | University of Aberdeen |
| ORCID | 0000-0002-0552-8096 |
| VERIFIED | Yes |
| TOTAL WORKS | 5 |
| TOTAL CITATIONS | 3 |
| WORKS AS AUTHOR | 5 |
| WORKS AS EDITOR | 0 |
| FIRST PUBLICATION YEAR | 1996 |
| LATEST PUBLICATION YEAR | 2024 |
| H-INDEX | 1 |
Common Flaws in Running Human Evaluation Experiments in NLP
While conducting a coordinated set of repeat runs of human evaluation experiments in NLP, we discovered flaws in every single experiment we selected for inclusion via a systematic process. In this squib, we describe the types of flaws we discovered, which include coding errors (e.g., loading the wrong system outputs to evaluate), failure to follow standard scientific practice (e.g., ad hoc exclusion of participants and responses), and mistakes in…
A Metrological Perspective on Reproducibility in NLP
Reproducibility has become an increasingly debated topic in NLP and ML over recent years, but so far, no commonly accepted definitions of even basic terms or concepts have emerged. The range of different definitions proposed within NLP/ML not only do not agree with each other, they are also not aligned with standard scientific definitions. This article examines the standard definitions of repeatability and reproducibility provided by the meta-sci…
That's Nice … What Can You Do With It
To which the answer was, in effect, "the brain isn't big enough yet to be used." The original aim of the CAM-Brain Project was to evolve behavior as complex as that of a kitten (the brain was going to control a robotic kitten, the "Robokoneko"). The functionality reported for the modules the brain was composed of was on the level of the XOR-function To date, no functionality appears to have been reported for the brain as a whole. 2 User-oriented …
An Investigation into the Validity of Some Metrics for Automatically Evaluating Natural Language Generation Systems
There is growing interest in using automatically computed corpus-based evaluation metrics to evaluate Natural Language Generation (NLG) systems, because these are often considerably cheaper than the human-based evaluations which have traditionally been used in NLG. We review previous work on NLG evaluation and on validation of automatic metrics in NLP, and then present the results of two studies of how well some metrics which are popular in other…
Richard Wagner
The drink, drink! I am poison'd.l These are Gertrude's dying words in Shakespeare's Hamlet. In a certain sense, play appears as a counterpart to Wagner's work, where it is always salvation that flows from sacred chalice. Before focusing on consequences of Wagner's oeuvre, I shall therefore briefly dwell on Shakespeare's play, which although written as early as turn of seventeenth century appears to criticize very images that explain continuing at…
An Investigation into the Validity of Some Metrics for Automatically Evaluating Natural Language Generation Systems
There is growing interest in using automatically computed corpus-based evaluation metrics to evaluate Natural Language Generation (NLG) systems, because these are often considerably cheaper than the human-based evaluations which have traditionally been used in NLG. We review previous work on NLG evaluation and on validation of automatic metrics in NLP, and then present the results of two studies of how well some metrics which are popular in other…
Richard Wagner
The drink, drink! I am poison'd.l These are Gertrude's dying words in Shakespeare's Hamlet. In a certain sense, play appears as a counterpart to Wagner's work, where it is always salvation that flows from sacred chalice. Before focusing on consequences of Wagner's oeuvre, I shall therefore briefly dwell on Shakespeare's play, which although written as early as turn of seventeenth century appears to criticize very images that explain continuing at…
That's Nice … What Can You Do With It
To which the answer was, in effect, "the brain isn't big enough yet to be used." The original aim of the CAM-Brain Project was to evolve behavior as complex as that of a kitten (the brain was going to control a robotic kitten, the "Robokoneko"). The functionality reported for the modules the brain was composed of was on the level of the XOR-function To date, no functionality appears to have been reported for the brain as a whole. 2 User-oriented …
An Investigation into the Validity of Some Metrics for Automatically Evaluating Natural Language Generation Systems
There is growing interest in using automatically computed corpus-based evaluation metrics to evaluate Natural Language Generation (NLG) systems, because these are often considerably cheaper than the human-based evaluations which have traditionally been used in NLG. We review previous work on NLG evaluation and on validation of automatic metrics in NLP, and then present the results of two studies of how well some metrics which are popular in other…
A Metrological Perspective on Reproducibility in NLP
Reproducibility has become an increasingly debated topic in NLP and ML over recent years, but so far, no commonly accepted definitions of even basic terms or concepts have emerged. The range of different definitions proposed within NLP/ML not only do not agree with each other, they are also not aligned with standard scientific definitions. This article examines the standard definitions of repeatability and reproducibility provided by the meta-sci…
Common Flaws in Running Human Evaluation Experiments in NLP
While conducting a coordinated set of repeat runs of human evaluation experiments in NLP, we discovered flaws in every single experiment we selected for inclusion via a systematic process. In this squib, we describe the types of flaws we discovered, which include coding errors (e.g., loading the wrong system outputs to evaluate), failure to follow standard scientific practice (e.g., ad hoc exclusion of participants and responses), and mistakes in…
Computer Science (4 works) · Natural Language Processing Techniques (4 works) · Topic Modeling (4 works) · Artificial Intelligence (3 works) · Natural language processing (3 works) · Information retrieval (2 works) · Machine learning (2 works) · Programming language (2 works) · Speech and dialogue systems (2 works) · Art (1 work)