Scoping Analytical Usability Evaluation Methods
A Case Study
Bibliographic Data
| ID | 15215974 |
|---|---|
| Authors | Ann Blandford (0000-0002-3198-7122, UCL Australia, corresponding author), Ann E Blandford (University College London), Joanne Hyde, Joanne K Hyde (Middlesex University), Thomas Green (0000-0002-3662-067X, University of Leeds), Thomas R G Green (University of Leeds), Iain Connell (University College London) |
| Year | 2008 |
| Volume | 23 |
| Issue | 3 |
| Pages | 278-327 |
| Publication date | 2008-08-29 |
| Peer Reviewed | Yes |
| Open Access | No |
| Type | ARTICLE |
| Venue | Human-Computer Interaction (JOURNAL) |
| Journal identifiers | ISSN: 0737-0024 • E-ISSN: 1532-7051 |
| Publisher | Taylor & Francis (PUBLISHER • GB) |
| DOI | 10.1080/07370020802278254 |
| OpenAlex | W2026566628 |
| Language | EN |
| References cited | 39 |
Analytical usability evaluation methods (UEMs) can complement empirical evaluation of systems: for example, they can often be used earlier in design and can provide accounts of why users might experience difficulties, as well as what those difficulties are. However, their properties and value are only partially understood. One way to improve our understanding is by detailed comparisons using a single interface or system as a target for evaluation, but we need to look deeper than simple problem counts: we need to consider what kinds of accounts each UEM offers, and why. Here, we report on a detailed comparison of eight analytical UEMs. These eight methods were applied to a robotic arm interface, and the findings were systematically compared against video data of the arm in use. The usability issues that were identified could be grouped into five categories: system design, user misconceptions, conceptual fit between user and system, physical issues, and contextual ones. Other possible categories such as user experience did not emerge in this particular study. With the exception of Heuristic Evaluation, which supported a range of insights, each analytical method was found to focus attention on just one or two categories of issues. Two of the three “home-grown” methods (Evaluating Multimodal Usability and Concept-based Analysis of Surface and Structural Misfits) were found to occupy particular niches in the space, whereas the third (Programmable User Modeling) did not. This approach has identified commonalities and contrasts between methods and provided accounts of why a particular method yielded the insights it did. Rather than considering measures such as problem count or thoroughness, this approach has yielded insights into the scope of each method
Cognitive walkthrough · Data science · Focus (optics · Heuristic evaluation · Human–computer interaction · Interface (matter · Usability · User interface · Computer Science · Human-Automation Interaction and Safety · Innovative Human-Technology Interaction · Usability and User Interface Design
What computers still can't do
Emotional design
Evaluating usability methods
The Nature of Device Models
Exploratory Sequential Data Analysis
Task-Action Grammars
An Overview of the Epic Architecture for Cognition and Performance With Application to Human-Computer Interaction
Repairing Damaged Merchandise
Representing Cognitive Activity in Complex Tasks
Syndetic Modelling
Project Ernestine
A semiotic definition of multimedia communication
| Citation velocity | historical |
|---|---|
| Highly cited | No |