Relational reasoning and generalization using non-symbolic neural networks

by   Atticus Geiger, et al.

Humans have a remarkable capacity to reason about abstract relational structures, an ability that may support some of the most impressive, human-unique cognitive feats. Because equality (or identity) is a simple and ubiquitous relational operator, equality reasoning has been a key case study for the broader question of abstract relational reasoning. This paper revisits the question of whether equality can be learned by neural networks that do not encode explicit symbolic structure. Earlier work arrived at a negative answer to this question, but that result holds only for a particular class of hand-crafted feature representations. In our experiments, we assess out-of-sample generalization of equality using both arbitrary representations and representations that have been pretrained on separate tasks to imbue them with abstract structure. In this setting, even simple neural networks are able to learn basic equality with relatively little training data. In a second case study, we show that sequential equality problems (learning ABA sequences) can be solved with only positive training instances. Finally, we consider a more complex, hierarchical equality problem, but this requires vastly more data. However, using a pretrained equality network as a modular component of this larger task leads to good performance with no task-specific training. Overall, these findings indicate that neural models are able to solve equality-based reasoning tasks, suggesting that essential aspects of symbolic reasoning can emerge from data-driven, non-symbolic learning processes.


page 1

page 2

page 3

page 4


Learning to Make Analogies by Contrasting Abstract Relational Structure

Analogical reasoning has been a principal focus of various waves of AI r...

Neural Abstract Reasoner

Abstract reasoning and logic inference are difficult problems for neural...

What Can Neural Networks Reason About?

Neural networks have successfully been applied to solving reasoning task...

A Solver-Free Framework for Scalable Learning in Neural ILP Architectures

There is a recent focus on designing architectures that have an Integer ...

What underlies rapid learning and systematic generalization in humans

Despite the groundbreaking successes of neural networks, contemporary mo...

Out-of-the box neural networks can support combinatorial generalization

Combinatorial generalization - the ability to understand and produce nov...

Break It Down: Evidence for Structural Compositionality in Neural Networks

Many tasks can be described as compositions over subroutines. Though mod...

Please sign up or login with your details

Forgot password? Click here to reset