An Investigation of Metrics for the In Situ Detection of Software Expertise
Bibliographic Data
| ID | 15216228 |
|---|---|
| Authors | Tovi Grossman (0000-0002-0494-5373, Autodesk (Canada), corresponding author), George Fitzmaurice (0000-0002-2834-7757, Autodesk (Canada)) |
| Year | 2014 |
| Volume | 30 |
| Issue | 1 |
| Pages | 64-102 |
| Publication date | 2014-01-30 |
| Open Access | No |
| Type | ARTICLE |
| Venue | Human-Computer Interaction (JOURNAL) |
| Journal identifiers | ISSN: 0737-0024 • E-ISSN: 1532-7051 |
| Publisher | Taylor & Francis (PUBLISHER • GB) |
| DOI | 10.1080/07370024.2014.881668 |
| OpenAlex | W2057144864 |
| Language | EN |
| References cited | 50 |
Task-based analysis is a common and effective way to measure expertise levels of software users. However, such assessments typically require in-person laboratory studies and inherently require knowledge of the user's task. Today, there is no accepted method for assessing a user's expertise levels outside of a lab, during a user's own home or work environment activities. In this article, we explore the feasibility of software applications automatically inferring a user's expertise levels, based on the user's in situ usage patterns. We outline the potential usage metrics that may be indicative of expertise levels and then perform a study, where we capture such metrics, by installing logging software in the participants' own workplace environments. We then invite those participants into a laboratory study and perform a more traditional task-based assessment of expertise. Our analysis of the study examines if metrics captured in situ, without any task knowledge, can be indicative of user expertise levels. The results show the existence of significant correlations between metrics calculated from in situ usage logs, and task-based user expertise assessments from our laboratory study. We discuss the implications of the results and how future software applications may be able to measure and leverage knowledge of the expertise of its users
In situ · Operating system · Chemistry · Computer Science · Intelligent Tutoring Systems and Adaptive Learning · Software Engineering Research · Software Engineering Techniques and Practices · Software
The Cambridge Handbook of Expertise and Expert Performance
Cognitive Tutors
The role of deliberate practice in the acquisition of expert performance.
Telling more than we can know
The Strategic Use of Complex Computer Systems
The Third Wave of Science Studies
Validity of psychological assessment
Expert and Exceptional Performance
| Highly cited | No |
|---|