Characterizing the Weight Space for Different Learning Models

by   Saurav Musunuru, et al.

Deep Learning has become one of the primary research areas in developing intelligent machines. Most of the well-known applications (such as Speech Recognition, Image Processing and NLP) of AI are driven by Deep Learning. Deep Learning algorithms mimic human brain using artificial neural networks and progressively learn to accurately solve a given problem. But there are significant challenges in Deep Learning systems. There have been many attempts to make deep learning models imitate the biological neural network. However, many deep learning models have performed poorly in the presence of adversarial examples. Poor performance in adversarial examples leads to adversarial attacks and in turn leads to safety and security in most of the applications. In this paper we make an attempt to characterize the solution space of a deep neural network in terms of three different subsets viz. weights belonging to exact trained patterns, weights belonging to generalized pattern set and weights belonging to adversarial pattern sets. We attempt to characterize the solution space with two seemingly different learning paradigms viz. the Deep Neural Networks and the Dense Associative Memory Model, which try to achieve learning via quite different mechanisms. We also show that adversarial attacks are generally less successful against Associative Memory Models than Deep Neural Networks.


page 2

page 3

page 4

page 5


Adversarial Examples in Deep Learning: Characterization and Divergence

The burgeoning success of deep learning has raised the security and priv...

Identifying Audio Adversarial Examples via Anomalous Pattern Detection

Audio processing models based on deep neural networks are susceptible to...

Towards Deep Learning Models Resistant to Adversarial Attacks

Recent work has demonstrated that neural networks are vulnerable to adve...

Defense against adversarial attacks on deep convolutional neural networks through nonlocal denoising

Despite substantial advances in network architecture performance, the su...

Adversarial Examples for Deep Learning Cyber Security Analytics

As advances in Deep Neural Networks demonstrate unprecedented levels of ...

Holistic Deep Learning

There is much interest in deep learning to solve challenges that arise i...

What do AI algorithms actually learn? - On false structures in deep learning

There are two big unsolved mathematical questions in artificial intellig...

Please sign up or login with your details

Forgot password? Click here to reset