A Data-Efficient Deep Learning Approach for Deployable Multimodal Social Robots

by   Heriberto Cuayáhuitl, et al.

The deep supervised and reinforcement learning paradigms (among others) have the potential to endow interactive multimodal social robots with the ability of acquiring skills autonomously. But it is still not very clear yet how they can be best deployed in real world applications. As a step in this direction, we propose a deep learning-based approach for efficiently training a humanoid robot to play multimodal games---and use the game of `Noughts & Crosses' with two variants as a case study. Its minimum requirements for learning to perceive and interact are based on a few hundred example images, a few example multimodal dialogues and physical demonstrations of robot manipulation, and automatic simulations. In addition, we propose novel algorithms for robust visual game tracking and for competitive policy learning with high winning rates, which substantially outperform DQN-based baselines. While an automatic evaluation shows evidence that the proposed approach can be easily extended to new games with competitive robot behaviours, a human evaluation with 130 humans playing with the Pepper robot confirms that highly accurate visual perception is required for successful game play.


page 2

page 6

page 12

page 15

page 18


Training an Interactive Humanoid Robot Using Multimodal Deep Reinforcement Learning

Training robots to perceive, act and communicate using multiple modaliti...

Sequence-based Multimodal Apprenticeship Learning For Robot Perception and Decision Making

Apprenticeship learning has recently attracted a wide attention due to i...

A Robot that Learns Connect Four Using Game Theory and Demonstrations

Teaching robots new skills using minimal time and effort has long been a...

A Robot's Expressive Language Affects Human Strategy and Perceptions in a Competitive Game

As robots are increasingly endowed with social and communicative capabil...

Hierarchical Policy Design for Sample-Efficient Learning of Robot Table Tennis Through Self-Play

Training robots with physical bodies requires developing new methods and...

SeekNet: Improved Human Instance Segmentation via Reinforcement Learning Based Optimized Robot Relocation

Amodal recognition is the ability of the system to detect occluded objec...

NimbRo Robots Winning RoboCup 2018 Humanoid AdultSize Soccer Competitions

Over the past few years, the Humanoid League rules have changed towards ...

Please sign up or login with your details

Forgot password? Click here to reset