What Artificial Neural Networks Can Tell Us About Human Language Acquisition

by   Alex Warstadt, et al.

Rapid progress in machine learning for natural language processing has the potential to transform debates about how humans learn language. However, the learning environments and biases of current artificial learners and humans diverge in ways that weaken the impact of the evidence obtained from learning simulations. For example, today's most effective neural language models are trained on roughly one thousand times the amount of linguistic data available to a typical child. To increase the relevance of learnability results from computational models, we need to train model learners without significant advantages over humans. If an appropriate model successfully acquires some target linguistic knowledge, it can provide a proof of concept that the target is learnable in a hypothesized human learning scenario. Plausible model learners will enable us to carry out experimental manipulations to make causal inferences about variables in the learning environment, and to rigorously test poverty-of-the-stimulus-style claims arguing for innate linguistic knowledge in humans on the basis of speculations about learnability. Comparable experiments will never be possible with human subjects due to practical and ethical considerations, making model learners an indispensable resource. So far, attempts to deprive current models of unfair advantages obtain sub-human results for key grammatical behaviors such as acceptability judgments. But before we can justifiably conclude that language learning requires more prior domain-specific knowledge than current models possess, we must first explore non-linguistic inputs in the form of multimodal stimuli and multi-agent interaction as ways to make our learners more efficient at learning from limited linguistic input.


page 4

page 6

page 7

page 13

page 15


Does Vision Accelerate Hierarchical Generalization of Neural Language Learners?

Neural language models (LMs) are arguably less data-efficient than human...

The cognitive roots of regularization in language

Regularization occurs when the output a learner produces is less variabl...

Developing Embodied Multisensory Dialogue Agents

A few decades of work in the AI field have focused efforts on developing...

Can neural networks acquire a structural bias from raw linguistic data?

We evaluate whether BERT, a widely used neural network for sentence proc...

Davinci the Dualist: the mind-body divide in large language models and in human learners

A large literature suggests that people are intuitive Dualists–they cons...

Learning Rewards from Linguistic Feedback

We explore unconstrained natural language feedback as a learning signal ...

DELAUNAY: a dataset of abstract art for psychophysical and machine learning research

Image datasets are commonly used in psychophysical experiments and in ma...

Please sign up or login with your details

Forgot password? Click here to reset