Multi Layer Neural Networks as Replacement for Pooling Operations

by   Wolfgang Fuhl, et al.

Pooling operations are a layer found in almost every modern neural network, which can be calculated at low cost and serves as a linear or nonlinear transfer function for data reduction. Many modern approaches have already dealt with replacing the common maximum value selection and mean value operations by others or even to provide a function that includes different functions which can be selected through changing parameters. Additional neural networks are used to estimate the parameters of these pooling functions. Therefore, these pooling layers need many additional parameters and increase the complexity of the whole model. In this work, we show that already one perceptron can be used very effectively as a pooling operation without increasing the complexity of the model. This kind of pooling allows to integrate multi-layer neural networks directly into a model as a pooling operation by restructuring the data and thus learning complex pooling operations. We compare our approach to tensor convolution with strides as a pooling operation and show that our approach is effective and reduces complexity. The restructuring of the data in combination with multiple perceptrons allows also to use our approach for upscaling, which is used for transposed convolutions in semantic segmentation.


page 1

page 2

page 3

page 4


Pooling Methods in Deep Neural Networks, a Review

Nowadays, Deep Neural Networks are among the main tools used in various ...

Convolutional Neural Networks Utilizing Multifunctional Spin-Hall MTJ Neurons

We propose a new network architecture for standard spin-Hall magnetic tu...

Structured Convolutions for Efficient Neural Network Design

In this work, we tackle model efficiency by exploiting redundancy in the...

Sorted Pooling in Convolutional Networks for One-shot Learning

We present generalized versions of the commonly used maximum pooling ope...

Improving Convolutional Neural Networks Via Conservative Field Regularisation and Integration

Current research in convolutional neural networks (CNN) focuses mainly o...

MorphPool: Efficient Non-linear Pooling Unpooling in CNNs

Pooling is essentially an operation from the field of Mathematical Morph...

Reinforcement Learning using Augmented Neural Networks

Neural networks allow Q-learning reinforcement learning agents such as d...

Please sign up or login with your details

Forgot password? Click here to reset