Generalisable Relational Reasoning With Comparators in Low-Dimensional Manifolds

by   Duo Wang, et al.

While modern deep neural architectures generalise well when test data is sampled from the same distribution as training data, they fail badly for cases when the test data distribution differs from the training distribution even along a few dimensions. This lack of out-of-distribution generalisation is increasingly manifested when the tasks become more abstract and complex, such as in relational reasoning. In this paper we propose a neuroscience-inspired inductive-biased module that can be readily amalgamated with current neural network architectures to improve out-of-distribution (o.o.d) generalisation performance on relational reasoning tasks. This module learns to project high-dimensional object representations to low-dimensional manifolds for more efficient and generalisable relational comparisons. We show that neural nets with this inductive bias achieve considerably better o.o.d generalisation performance for a range of relational reasoning tasks. We finally analyse the proposed inductive bias module to understand the importance of lower dimension projection, and propose an augmentation to the algorithmic alignment theory to better measure algorithmic alignment with generalisation.


Dilated DenseNets for Relational Reasoning

Despite their impressive performance in many tasks, deep neural networks...

Relational inductive bias for physical construction in humans and machines

While current deep learning systems excel at tasks such as object classi...

On Neural Architecture Inductive Biases for Relational Tasks

Current deep learning approaches have shown good in-distribution general...

R-SQAIR: Relational Sequential Attend, Infer, Repeat

Traditional sequential multi-object attention models rely on a recurrent...

Neural Algorithmic Reasoning with Causal Regularisation

Recent work on neural algorithmic reasoning has investigated the reasoni...

The Relational Bottleneck as an Inductive Bias for Efficient Abstraction

A central challenge for cognitive science is to explain how abstract con...

V-PROM: A Benchmark for Visual Reasoning Using Visual Progressive Matrices

One of the primary challenges faced by deep learning is the degree to wh...

Please sign up or login with your details

Forgot password? Click here to reset