Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses

by   Jacob Granley, et al.

Neuroprostheses show potential in restoring lost sensory function and enhancing human capabilities, but the sensations produced by current devices often seem unnatural or distorted. Exact placement of implants and differences in individual perception lead to significant variations in stimulus response, making personalized stimulus optimization a key challenge. Bayesian optimization could be used to optimize patient-specific stimulation parameters with limited noisy observations, but is not feasible for high-dimensional stimuli. Alternatively, deep learning models can optimize stimulus encoding strategies, but typically assume perfect knowledge of patient-specific variations. Here we propose a novel, practically feasible approach that overcomes both of these fundamental limitations. First, a deep encoder network is trained to produce optimal stimuli for any individual patient by inverting a forward model mapping electrical stimuli to visual percepts. Second, a preferential Bayesian optimization strategy utilizes this encoder to optimize patient-specific parameters for a new patient, using a minimal number of pairwise comparisons between candidate stimuli. We demonstrate the viability of this approach on a novel, state-of-the-art visual prosthesis model. We show that our approach quickly learns a personalized stimulus encoder, leads to dramatic improvements in the quality of restored vision, and is robust to noisy patient feedback and misspecifications in the underlying forward model. Overall, our results suggest that combining the strengths of deep learning and Bayesian optimization could significantly improve the perceptual experience of patients fitted with visual prostheses and may prove a viable solution for a range of neuroprosthetic technologies.


page 3

page 7

page 8

page 19


A Hybrid Neural Autoencoder for Sensory Neuroprostheses and Its Applications in Bionic Vision

Sensory neuroprostheses are emerging as a promising technology to restor...

Bayesian optimization for automatic design of face stimuli

Investigating the cognitive and neural mechanisms involved with face pro...

Personalized visual encoding model construction with small data

Encoding models that predict brain response patterns to stimuli are one ...

High-dimensional Bayesian Optimization of Personalized Cardiac Model Parameters via an Embedded Generative Model

The estimation of patient-specific tissue properties in the form of mode...

ConBO: Conditional Bayesian Optimization

Bayesian optimization is a class of data efficient model based algorithm...

Generative Melody Composition with Human-in-the-Loop Bayesian Optimization

Deep generative models allow even novice composers to generate various m...

Arachnophobia Exposure Therapy using Experience-driven Procedural Content Generation via Reinforcement Learning (EDPCGRL)

Personalized therapy, in which a therapeutic practice is adapted to an i...

Please sign up or login with your details

Forgot password? Click here to reset