Experimentally realized memristive memory augmented neural network

by   Ruibin Mao, et al.

Lifelong on-device learning is a key challenge for machine intelligence, and this requires learning from few, often single, samples. Memory augmented neural network has been proposed to achieve the goal, but the memory module has to be stored in an off-chip memory due to its size. Therefore the practical use has been heavily limited. Previous works on emerging memory-based implementation have difficulties in scaling up because different modules with various structures are difficult to integrate on the same chip and the small sense margin of the content addressable memory for the memory module heavily limited the degree of mismatch calculation. In this work, we implement the entire memory augmented neural network architecture in a fully integrated memristive crossbar platform and achieve an accuracy that closely matches standard software on digital hardware for the Omniglot dataset. The successful demonstration is supported by implementing new functions in crossbars in addition to widely reported matrix multiplications. For example, the locality-sensitive hashing operation is implemented in crossbar arrays by exploiting the intrinsic stochasticity of memristor devices. Besides, the content-addressable memory module is realized in crossbars, which also supports the degree of mismatches. Simulations based on experimentally validated models show such an implementation can be efficiently scaled up for one-shot learning on the Mini-ImageNet dataset. The successful demonstration paves the way for practical on-device lifelong learning and opens possibilities for novel attention-based algorithms not possible in conventional hardware.


page 12

page 19

page 22

page 27

page 29

page 30

page 34

page 37


On-chip learning in a conventional silicon MOSFET based Analog Hardware Neural Network

On-chip learning in a crossbar array based analog hardware Neural Networ...

Bulk-Switching Memristor-based Compute-In-Memory Module for Deep Neural Network Training

The need for deep neural network (DNN) models with higher performance an...

Implementation of a Binary Neural Network on a Passive Array of Magnetic Tunnel Junctions

The increasing scale of neural networks and their growing application sp...

In-Memory Nearest Neighbor Search with FeFET Multi-Bit Content-Addressable Memories

Nearest neighbor (NN) search is an essential operation in many applicati...

Sequence Learning and Consolidation on Loihi using On-chip Plasticity

In this work we develop a model of predictive learning on neuromorphic h...

Device Modeling Bias in ReRAM-based Neural Network Simulations

Data-driven modeling approaches such as jump tables are promising techni...

Please sign up or login with your details

Forgot password? Click here to reset