Non-verbal Communication with a Social Robot Peer: Towards Robot Assisted Interactive Sign Language Tutoring

Akalin N., Uluer P., Köse H.

14th IEEE-RAS International Conference on Humanoid Robots (Humanoids), Madrid, İspanya, 18 - 20 Kasım 2014, ss.1122-1127 identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Basıldığı Şehir: Madrid
  • Basıldığı Ülke: İspanya
  • Sayfa Sayıları: ss.1122-1127
  • Anahtar Kelimeler: Humanoid robots, interaction games, non-verbal communication, sign language
  • Galatasaray Üniversitesi Adresli: Evet

Özet

This paper presents a humanoid robot assisted imitation based interactive game for Sign Language (SL) tutoring. The game is specially designed for children with communication impairments. The work presented is a part of Robot Sign Language Tutor* project. In this study, a Robovie R3 humanoid robot is used to express a set of chosen words in SL using hand movements, body and face gestures. The robot platform is specially modified with LEDs in face (for non-manual facial gestures), additional DoFs in wrist and 5 independent fingers at hands for robust SL generation. The robot is able to communicate with the participant by recognizing signs and colored flashcards through a RGB-D camera based system and generating a selected subset of signs, including motivating facial gestures, in return. The game also aims to improve children's imitation and turn-taking skills and to teach the words semantically. Current paper presents results from the preliminary study with a group of hearing impaired children, where children had almost 100% score in recognizing the robot's signs from a subset of 12 words.