I am Robot: Neuromuscular Reinforcement Learning to Actuate Human Limbs through Functional Electrical Stimulation

by   Nat Wannawas, et al.

Human movement disorders or paralysis lead to the loss of control of muscle activation and thus motor control. Functional Electrical Stimulation (FES) is an established and safe technique for contracting muscles by stimulating the skin above a muscle to induce its contraction. However, an open challenge remains on how to restore motor abilities to human limbs through FES, as the problem of controlling the stimulation is unclear. We are taking a robotics perspective on this problem, by developing robot learning algorithms that control the ultimate humanoid robot, the human body, through electrical muscle stimulation. Human muscles are not trivial to control as actuators due to their force production being non-stationary as a result of fatigue and other internal state changes, in contrast to robot actuators which are well-understood and stationary over broad operation ranges. We present our Deep Reinforcement Learning approach to the control of human muscles with FES, using a recurrent neural network for dynamic state representation, to overcome the unobserved elements of the behaviour of human muscles under external stimulation. We demonstrate our technique both in neuromuscular simulations but also experimentally on a human. Our results show that our controller can learn to manipulate human muscles, applying appropriate levels of stimulation to achieve the given tasks while compensating for advancing muscle fatigue which arises throughout the tasks. Additionally, our technique can learn quickly enough to be implemented in real-world human-in-the-loop settings.


page 1

page 5


Neuromuscular Reinforcement Learning to Actuate Human Limbs through FES

Functional Electrical Stimulation (FES) is a technique to evoke muscle c...

Belousov-Zhabotinsky liquid marbles in robot control

We show how to control the movement of a wheeled robot using on-board li...

Structure Learning in Motor Control:A Deep Reinforcement Learning Model

Motor adaptation displays a structure-learning effect: adaptation to a n...

A Framework for Learning from Demonstration with Minimal Human Effort

We consider robot learning in the context of shared autonomy, where cont...

Mechatronic Design of a Dribbling System for RoboCup Small Size Robot

RoboCup SSL is an excellent platform for researching artificial intellig...

Towards AI-controlled FES-restoration of arm movements: neuromechanics-based reinforcement learning for 3-D reaching

Reaching disabilities affect the quality of life. Functional Electrical ...

From hand to brain and back: Grip forces deliver insight into the functional plasticity of somatosensory processes

The human somatosensory cortex is intimately linked to other central bra...

Please sign up or login with your details

Forgot password? Click here to reset