Concerning Trends in Likert Scale Usage in Human-robot Interaction

Towards Improving Best Practices

Bibliographic Data

ID22190688
AuthorsMariah Schrum (0000-0001-6277-4720, Georgia Institute of Technology), Muyleng Ghuy (Georgia Institute of Technology), Erin Hedlund-Botti (0000-0001-9890-4873, Georgia Institute of Technology), Manisha Natarajan (0000-0001-7583-9685, Georgia Institute of Technology), Michael Johnson (0009-0006-4780-3136, Georgia Institute of Technology), Matthew Gombolay (0000-0002-5321-6038, Georgia Institute of Technology)
Year2023
Volume12
Issue3
Pages1-32
Publication date2023-09-30
Open AccessYes
TypeARTICLE
VenueACM Transactions on Human-Robot Interaction (JOURNAL)
Journal identifiersISSN: 2573-9522 • E-ISSN: 2573-9522
PublisherAssociation for Computing Machinery (ACM) (PUBLISHER)
DOI10.1145/3572784
OpenAlexW4309698321
LanguageEN
Citations received7
References cited90

As robots become more prevalent, the importance of the field of human-robot interaction (HRI) grows accordingly. As such, we should endeavor to employ the best statistical practices in HRI research. Likert scales are commonly used metrics in HRI to measure perceptions and attitudes. Due to misinformation or honest mistakes, many HRI researchers do not adopt best practices when analyzing Likert data. We conduct a review of psychometric literature to determine the current standard for Likert scale design and analysis. Next, we conduct a survey of five years of the International Conference on Human-Robot Interaction (HRIc) (2016 through 2020) and report on incorrect statistical practices and design of Likert scales [ 1 , 2 , 3 , 5 , 7 ]. During these years, only 4 of the 144 papers applied proper statistical testing to correctly designed Likert scales. We additionally conduct a survey of best practices across several venues and provide a comparative analysis to determine how Likert practices differ across the field of Human-robot Interaction. We find that a venue’s impact score negatively correlates with number of Likert-related errors and acceptance rate, and total number of papers accepted per venue positively correlates with the number of errors. We also find statistically significant differences between venues for the frequency of misnomer and design errors. Our analysis suggests there are areas for meaningful improvement in the design and testing of Likert scales. Based on our findings, we provide guidelines and a tutorial for researchers for developing and analyzing Likert scales and associated data. We also detail a list of recommendations to improve the accuracy of conclusions drawn from Likert data

Data science · Likert scale · Statistics · Applied Psychology · Behavioral Health and Interventions · Computer Science · Human-Automation Interaction and Safety · Mathematics · Occupational Health and Safety Research · Psychology

Tools

Open DOI