Malicious Network Traffic Detection via Deep Learning: An Information Theoretic View

by   Erick Galinkin, et al.

The attention that deep learning has garnered from the academic community and industry continues to grow year over year, and it has been said that we are in a new golden age of artificial intelligence research. However, neural networks are still often seen as a "black box" where learning occurs but cannot be understood in a human-interpretable way. Since these machine learning systems are increasingly being adopted in security contexts, it is important to explore these interpretations. We consider an Android malware traffic dataset for approaching this problem. Then, using the information plane, we explore how homeomorphism affects learned representation of the data and the invariance of the mutual information captured by the parameters on that data. We empirically validate these results, using accuracy as a second measure of similarity of learned representations. Our results suggest that although the details of learned representations and the specific coordinate system defined over the manifold of all parameters differ slightly, the functional approximations are the same. Furthermore, our results show that since mutual information remains invariant under homeomorphism, only feature engineering methods that alter the entropy of the dataset will change the outcome of the neural network. This means that for some datasets and tasks, neural networks require meaningful, human-driven feature engineering or changes in architecture to provide enough information for the neural network to generate a sufficient statistic. Applying our results can serve to guide analysis methods for machine learning engineers and suggests that neural networks that can exploit the convolution theorem are equally accurate as standard convolutional neural networks, and can be more computationally efficient.


page 35

page 37


Mutual information estimation for graph convolutional neural networks

Measuring model performance is a key issue for deep learning practitione...

Analytic Mutual Information in Bayesian Neural Networks

Bayesian neural networks have successfully designed and optimized a robu...

Inverting Supervised Representations with Autoregressive Neural Density Models

Understanding the nature of representations learned by supervised machin...

Explaining Representation by Mutual Information

Science is used to discover the law of world. Machine learning can be us...

Making Sense of CNNs: Interpreting Deep Representations Their Invariances with INNs

To tackle increasingly complex tasks, it has become an essential ability...

Softmax Is Not an Artificial Trick: An Information-Theoretic View of Softmax in Neural Networks

Despite great popularity of applying softmax to map the non-normalised o...

Doing the impossible: Why neural networks can be trained at all

As deep neural networks grow in size, from thousands to millions to bill...

Please sign up or login with your details

Forgot password? Click here to reset