Experience with an Affective Robot Assistant for Children with Hearing Disabilities

Creative Commons License

ULUER P., Köse H., Gumuslu E., EROL BARKANA D.

INTERNATIONAL JOURNAL OF SOCIAL ROBOTICS, cilt.15, sa.4, ss.643-660, 2023 (SCI-Expanded) identifier identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 15 Sayı: 4
  • Basım Tarihi: 2023
  • Doi Numarası: 10.1007/s12369-021-00830-5
  • Dergi Adı: INTERNATIONAL JOURNAL OF SOCIAL ROBOTICS
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Applied Science & Technology Source, Compendex, Computer & Applied Sciences, INSPEC, Psycinfo
  • Sayfa Sayıları: ss.643-660
  • Anahtar Kelimeler: Social robots, Human-robot interaction, Machine learning, Deep learning, Emotion recognition, Physiological signals, EMOTION, RECOGNITION, SIGNALS, AUTISM
  • Galatasaray Üniversitesi Adresli: Evet

Özet

This study presents an assistive robotic system enhanced with emotion recognition capabilities for children with hearing disabilities. The system is designed and developed for the audiometry tests and rehabilitation of children in a clinical setting and includes a social humanoid robot (Pepper), an interactive interface, gamified audiometry tests, sensory setup and a machine/deep learning based emotion recognition module. Three scenarios involving conventional setup, tablet setup and setup with the robot+tablet are evaluated with 16 children having cochlear implant or hearing aid. Several machine learning techniques and deep learning models are used for the classification of the three test setups and for the classification of the emotions (pleasant, neutral, unpleasant) of children using the recorded physiological signals by E4 wristband. The results show that the collected signals during the tests can be separated successfully and the positive and negative emotions of children can be better distinguished when they interact with the robot than in the other two setups. In addition, the children's objective and subjective evaluations as well as their impressions about the robot and its emotional behaviors are analyzed and discussed extensively.