Image-Based Tactile Emojis

Improved Interpretation of Message Intention and Subtle Nuance for Visually Impaired Individuals

Bibliographic Data

ID15216231
AuthorsYuri Choi (0000-0002-7704-4269, Korea Advanced Institute of Science and Technology, corresponding author), Kyung Hoon Hyun (0000-0001-6379-9700, Hanyang University, Korea), Ji-Hyun Lee (0000-0001-6420-5150, Korea Advanced Institute of Science and Technology)
Year2017
Volume35
Issue1
Pages40-69
Publication date2017-05-12
Peer ReviewedYes
Open AccessNo
TypeARTICLE
VenueHuman-Computer Interaction (JOURNAL)
Journal identifiersISSN: 0737-0024 • E-ISSN: 1532-7051
PublisherTaylor & Francis (PUBLISHER • GB)
DOI10.1080/07370024.2017.1324305
OpenAlexW2613627258
LanguageEN
Citations received1
References cited31

To enhance missing nonverbal cues in computer-mediated communication using text, those who can see often use emojis or emoticons. Although emojis for the sighted have transformed throughout the years to animated forms and added sound effects, emojis for visually impaired people remain underdeveloped. This study tested how tactile emojis based on visual imagery combined with the Braille system can enhance clarity in the computer-mediated communication environment for those with visual impairments. Results of this study confirmed three things: Visually impaired subjects were able to connect emotional emojis to the emotion they represented without any prior guidance, image-based (picture-based) and non-image-based (abstraction-based) tactile emoji were equally learnable, and the clarity of intended meaning was improved when an emoji was used with text (Braille). Thirty visually impaired subjects were able to match an average of 67% of emotions without prior guidance, and three of the four subjects who matched perfectly both before and after guidance were congenitally blind. The subjects had the most trouble discriminating the facial feature of “fear” between “sadness” or “surprised” for they shared similar traits. After guidance, the image-based tactile design elicited an average of 81% correct answers, whereas the non-image-based tactile design elicited an average of 37%, showing that the image-based tactile design was more effective for learning the meaning of emojis. The clarity of the sentence was also improved. This study shows that image-based tactile emojis can improve the texting experience of visually impaired individuals to a level where they can communicate subtle emotional cues through tactile imagery. This advance could minimize the service gap between sighted and visually impaired people and offer a much more abundant computer-mediated communication environment for visually impaired individuals

Anger · Braille · CLARITY · Cognitive psychology · Emoji · Human–computer interaction · Meaning (existential · Nonverbal communication · Sadness · Visually impaired · Communication · Computer Science · Digital Communication and Language · Media, Communication, and Education · Multisensory perception and integration · Psychology · Social Psychology

Tools

Open DOISci-Hub