Episodic memory for continual model learning

by   David G. Nagy, et al.

Both the human brain and artificial learning agents operating in real-world or comparably complex environments are faced with the challenge of online model selection. In principle this challenge can be overcome: hierarchical Bayesian inference provides a principled method for model selection and it converges on the same posterior for both off-line (i.e. batch) and online learning. However, maintaining a parameter posterior for each model in parallel has in general an even higher memory cost than storing the entire data set and is consequently clearly unfeasible. Alternatively, maintaining only a limited set of models in memory could limit memory requirements. However, sufficient statistics for one model will usually be insufficient for fitting a different kind of model, meaning that the agent loses information with each model change. We propose that episodic memory can circumvent the challenge of limited memory-capacity online model selection by retaining a selected subset of data points. We design a method to compute the quantities necessary for model selection even when the data is discarded and only statistics of one (or few) learnt models are available. We demonstrate on a simple model that a limited-sized episodic memory buffer, when the content is optimised to retain data with statistics not matching the current representation, can resolve the fundamental challenge of online model selection.


page 1

page 2

page 3

page 4


Bayesian Model Selection for a Class of Spatially-Explicit Capture Recapture Models

A vast amount of ecological knowledge generated recently has hinged upon...

Robust and Parallel Bayesian Model Selection

Effective and accurate model selection is an important problem in modern...

Robust and Reproducible Model Selection Using Bagged Posteriors

Bayesian model selection is premised on the assumption that the data are...

Anytime Model Selection in Linear Bandits

Model selection in the context of bandit optimization is a challenging p...

Toward a Universal Cortical Algorithm: Examining Hierarchical Temporal Memory in Light of Frontal Cortical Function

A wide range of evidence points toward the existence of a common algorit...

Model Selection Techniques -- An Overview

In the era of big data, analysts usually explore various statistical mod...

Robust Model Selection with Application in Single-Cell Multiomics Data

Model selection is critical in the modern statistics and machine learnin...

Please sign up or login with your details

Forgot password? Click here to reset