Is Bio-Inspired Learning Better than Backprop? Benchmarking Bio Learning vs. Backprop

by   Manas Gupta, et al.

Bio-inspired learning has been gaining popularity recently given that Backpropagation (BP) is not considered biologically plausible. Many algorithms have been proposed in the literature which are all more biologically plausible than BP. However, apart from overcoming the biological implausibility of BP, a strong motivation for using Bio-inspired algorithms remains lacking. In this study, we undertake a holistic comparison of BP vs. multiple Bio-inspired algorithms to answer the question of whether Bio-learning offers additional benefits over BP, rather than just biological plausibility. We test Bio-algorithms under different design choices such as access to only partial training data, resource constraints in terms of the number of training epochs, sparsification of the neural network parameters and addition of noise to input samples. Through these experiments, we notably find two key advantages of Bio-algorithms over BP. Firstly, Bio-algorithms perform much better than BP when the entire training dataset is not supplied. Four of the five Bio-algorithms tested outperform BP by upto 5 training dataset is available. Secondly, even when the full dataset is available, Bio-algorithms learn much quicker and converge to a stable accuracy in far lesser training epochs than BP. Hebbian learning, specifically, is able to learn in just 5 epochs compared to around 100 epochs required by BP. These insights present practical reasons for utilising Bio-learning rather than just its biological plausibility and also point towards interesting new directions for future work on Bio-learning.


page 1

page 6


Unlocking the Potential of Similarity Matching: Scalability, Supervision and Pre-training

While effective, the backpropagation (BP) algorithm exhibits limitations...

Biologically-primed deep neural network improves colorectal Cancer Molecular subtypes prediction from H E stained images

Colorectal cancer (CRC) molecular subtypes play a crucial role in determ...

The expected performance of stellar parametrization with Gaia spectrophotometry

Gaia will obtain astrometry and spectrophotometry for essentially all so...

Constrained Parameter Inference as a Principle for Learning

Learning in biological and artificial neural networks is often framed as...

Biologically-plausible learning algorithms can scale to large datasets

The backpropagation (BP) algorithm is often thought to be biologically i...

Understanding Predictive Coding as an Adaptive Trust-Region Method

Predictive coding (PC) is a brain-inspired local learning algorithm that...

Dualities in Tree Representations

A characterization of the tree T^* such that BP(T^*)=DFUDS(T), the rever...

Please sign up or login with your details

Forgot password? Click here to reset