Training an Interactive Humanoid Robot Using Multimodal Deep Reinforcement Learning

by   Heriberto Cuayáhuitl, et al.
University of Lincoln

Training robots to perceive, act and communicate using multiple modalities still represents a challenging problem, particularly if robots are expected to learn efficiently from small sets of example interactions. We describe a learning approach as a step in this direction, where we teach a humanoid robot how to play the game of noughts and crosses. Given that multiple multimodal skills can be trained to play this game, we focus our attention to training the robot to perceive the game, and to interact in this game. Our multimodal deep reinforcement learning agent perceives multimodal features and exhibits verbal and non-verbal actions while playing. Experimental results using simulations show that the robot can learn to win or draw up to 98 test of the proposed multimodal system for the targeted game---integrating speech, vision and gestures---reports that reasonable and fluent interactions can be achieved using the proposed approach.


page 2

page 3

page 4


A Data-Efficient Deep Learning Approach for Deployable Multimodal Social Robots

The deep supervised and reinforcement learning paradigms (among others) ...

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning

We investigate whether Deep Reinforcement Learning (Deep RL) is able to ...

Robot gains Social Intelligence through Multimodal Deep Reinforcement Learning

For robots to coexist with humans in a social world like ours, it is cru...

A Robot that Learns Connect Four Using Game Theory and Demonstrations

Teaching robots new skills using minimal time and effort has long been a...

An End-to-End Human Simulator for Task-Oriented Multimodal Human-Robot Collaboration

This paper proposes a neural network-based user simulator that can provi...

DAQN: Deep Auto-encoder and Q-Network

The deep reinforcement learning method usually requires a large number o...

Please sign up or login with your details

Forgot password? Click here to reset