Modeling Item Response Theory with Stochastic Variational Inference

by   Mike Wu, et al.

Item Response Theory (IRT) is a ubiquitous model for understanding human behaviors and attitudes based on their responses to questions. Large modern datasets offer opportunities to capture more nuances in human behavior, potentially improving psychometric modeling leading to improved scientific understanding and public policy. However, while larger datasets allow for more flexible approaches, many contemporary algorithms for fitting IRT models may also have massive computational demands that forbid real-world application. To address this bottleneck, we introduce a variational Bayesian inference algorithm for IRT, and show that it is fast and scalable without sacrificing accuracy. Applying this method to five large-scale item response datasets from cognitive science and education yields higher log likelihoods and higher accuracy in imputing missing data than alternative inference algorithms. Using this new inference approach we then generalize IRT with expressive Bayesian models of responses, leveraging recent advances in deep learning to capture nonlinear item characteristic curves (ICC) with neural networks. Using an eigth-grade mathematics test from TIMSS, we show our nonlinear IRT models can capture interesting asymmetric ICCs. The algorithm implementation is open-source, and easily usable.


page 1

page 2

page 3

page 4


Variational Item Response Theory: Fast, Accurate, and Expressive

Item Response Theory is a ubiquitous algorithm used around the world to ...

Flexible Bayesian modelling in dichotomous item response theory using mixtures of skewed item curves

Most Item Response Theory (IRT) models for dichotomous responses are bas...

Assessment meets Learning: On the relation between Item Response Theory and Bayesian Knowledge Tracing

Few models have been more ubiquitous in their respective fields than Bay...

Curve Fitting from Probabilistic Emissions and Applications to Dynamic Item Response Theory

Item response theory (IRT) models are widely used in psychometrics and e...

Automated Scoring of Graphical Open-Ended Responses Using Artificial Neural Networks

Automated scoring of free drawings or images as responses has yet to be ...

Bayesian item response models for citizen science ecological data

So-called citizen science data elicited from crowds has become increasin...

Learning Latent Parameters without Human Response Patterns: Item Response Theory with Artificial Crowds

Incorporating Item Response Theory (IRT) into NLP tasks can provide valu...

Please sign up or login with your details

Forgot password? Click here to reset