A Multi-In and Multi-Out Dendritic Neuron Model and its Optimization

by   Yu Ding, et al.

Artificial neural networks (ANNs), inspired by the interconnection of real neurons, have achieved unprecedented success in various fields such as computer vision and natural language processing. Recently, a novel mathematical ANN model, known as the dendritic neuron model (DNM), has been proposed to address nonlinear problems by more accurately reflecting the structure of real neurons. However, the single-output design limits its capability to handle multi-output tasks, significantly lowering its applications. In this paper, we propose a novel multi-in and multi-out dendritic neuron model (MODN) to tackle multi-output tasks. Our core idea is to introduce a filtering matrix to the soma layer to adaptively select the desired dendrites to regress each output. Because such a matrix is designed to be learnable, MODN can explore the relationship between each dendrite and output to provide a better solution to downstream tasks. We also model a telodendron layer into MODN to simulate better the real neuron behavior. Importantly, MODN is a more general and unified framework that can be naturally specialized as the DNM by customizing the filtering matrix. To explore the optimization of MODN, we investigate both heuristic and gradient-based optimizers and introduce a 2-step training method for MODN. Extensive experimental results performed on 11 datasets on both binary and multi-class classification tasks demonstrate the effectiveness of MODN, with respect to accuracy, convergence, and generality.


Single neuron-based neural networks are as efficient as dense deep neural networks in binary and multi-class recognition problems

Recent advances in neuroscience have revealed many principles about neur...

Non-linear Neurons with Human-like Apical Dendrite Activations

In order to classify linearly non-separable data, neurons are typically ...

Towards the Evolution of Multi-Layered Neural Networks: A Dynamic Structured Grammatical Evolution Approach

Current grammar-based NeuroEvolution approaches have several shortcoming...

One Neuron Saved Is One Neuron Earned: On Parametric Efficiency of Quadratic Networks

Inspired by neuronal diversity in the biological neural system, a pletho...

Operational Neural Networks

Feed-forward, fully-connected Artificial Neural Networks (ANNs) or the s...

Nucleus Neural Network for Super Robust Learning

Artificial neural networks which model the neurons and connecting archit...

A Novel Neuron Model of Visual Processor

Simulating and imitating the neuronal network of humans or mammals is a ...

Please sign up or login with your details

Forgot password? Click here to reset