Identification of Low-engaged Learners in Robot-led Second Language Conversations with Adults

The main aim of this study is to investigate if verbal, vocal, and facial information can be used to identify low-engaged second language learners in robot-led conversation practice. The experiments were performed on voice recordings and video data from 50 conversations, in which a robotic head talks with pairs of adult language learners using four different interaction strategies with varying robot-learner focus and initiative. It was found that these robot interaction strategies influenced learner activity and engagement. The verbal analysis indicated that learners with low activity rated the robot significantly lower on two out of four scales related to social competence. The acoustic vocal and video-based facial analysis, based on manual annotations or machine learning classification, both showed that learners with low engagement rated the robot’s social competencies consistently, and in several cases significantly, lower, and in addition rated the learning effectiveness lower. The agreement between manual and automatic identification of low-engaged learners based on voice recordings or face videos was further found to be adequate for future use. These experiments constitute a first step towards enabling adaption to learners’ activity and engagement through within- and between-strategy changes of the robot’s interaction with learners

Competence (human resources · Conversation · Human–computer interaction · Identification (biology · Mobile robot · Multimedia · Robot · Social robot · AI in Service Interactions · Computer Science · Psychology · Social Robot Interaction and HRI · Speech and dialogue systems · Artificial Intelligence · Communication · Social Psychology

• A Survey of Multimodal Perception Methods for Human–Robot Interaction in Social Environments

  Open Access•John A Duncan, Farshid Alambeigi et al.•ACM Transactions on Human-Robot…•2024

  Añadir item

• A Two-Month Field Trial in an Elementary School for Long-Term Human–Robot Interaction

  Open Access•Takayuki Kanda, Ryuya Sato et al.•IEEE Transactions on Robotics•2007

  Añadir item

• The Measurement of Observer Agreement for Categorical Data

  J R Landis, Gary G Koch•Biometrics•1977

  Añadir item

• Robot Interaction Styles for Conversation Practice in Second Language Learning

  Open Access•Olov Engwall, José Lopes et al.•International Journal of Social…•2020

  Añadir item

• On-the-Fly Detection of User Engagement Decrease in Spontaneous Human–Robot Interaction Using Recurrent and Deep Neural Networks

  Open Access•Atef Ben-Youssef, Giovanna Varni et al.•International Journal of Social…•2019

  Añadir item

• Automatically Classifying User Engagement for Dynamic Multi-party Human–Robot Interaction

  Open Access•Mary Ellen Foster, Andre Gaschler et al.•International Journal of Social…•2017

  Añadir item