Human-Level Control through Directly-Trained Deep Spiking Q-Networks

by   Guisong Liu, et al.
Zhejiang University
Southwestern University of Finance and Economics
University of Electronic Science and Technology of China

As the third-generation neural networks, Spiking Neural Networks (SNNs) have great potential on neuromorphic hardware because of their high energy-efficiency. However, Deep Spiking Reinforcement Learning (DSRL), i.e., the Reinforcement Learning (RL) based on SNNs, is still in its preliminary stage due to the binary output and the non-differentiable property of the spiking function. To address these issues, we propose a Deep Spiking Q-Network (DSQN) in this paper. Specifically, we propose a directly-trained deep spiking reinforcement learning architecture based on the Leaky Integrate-and-Fire (LIF) neurons and Deep Q-Network (DQN). Then, we adapt a direct spiking learning algorithm for the Deep Spiking Q-Network. We further demonstrate the advantages of using LIF neurons in DSQN theoretically. Comprehensive experiments have been conducted on 17 top-performing Atari games to compare our method with the state-of-the-art conversion method. The experimental results demonstrate the superiority of our method in terms of performance, stability, robustness and energy-efficiency. To the best of our knowledge, our work is the first one to achieve state-of-the-art performance on multiple Atari games with the directly-trained SNN.


Deep Reinforcement Learning with Spiking Q-learning

With the help of special neuromorphic hardware, spiking neural networks ...

Spiking Deep Residual Network

Recently, spiking neural network (SNN) has received significant attentio...

Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networks

Spiking neural networks (SNNs) have great potential for energy-efficient...

Reinforcement Learning with Low-Complexity Liquid State Machines

We propose reinforcement learning on simple networks consisting of rando...

Towards deep learning with spiking neurons in energy based models with contrastive Hebbian plasticity

In machine learning, error back-propagation in multi-layer neural networ...

Modeling Associative Plasticity between Synapses to Enhance Learning of Spiking Neural Networks

Spiking Neural Networks (SNNs) are the third generation of artificial ne...

Please sign up or login with your details

Forgot password? Click here to reset