Fast Model-Selection through Adapting Design of Experiments Maximizing Information Gain
share
To perform model-selection efficiently, we must run informative experiments. Here, we extend a seminal method for designing Bayesian optimal experiments that maximize the information gained from data collected. We introduce two computational improvements: a search algorithm from artificial intelligence and a sampling procedure shrinking the space of possible experiments to evaluate. We collected data for five different experimental designs and show that experiments optimized for information gain make model-selection faster, and cheaper, as compared to the designs chosen by a pool of expert experimentalists. Our procedure is general and can be applied iteratively to lab, field and online experiments.
READ FULL TEXT