Bootstrapping Adaptive Human-Machine Interfaces with Offline Reinforcement Learning

by   Jensen Gao, et al.

Adaptive interfaces can help users perform sequential decision-making tasks like robotic teleoperation given noisy, high-dimensional command signals (e.g., from a brain-computer interface). Recent advances in human-in-the-loop machine learning enable such systems to improve by interacting with users, but tend to be limited by the amount of data that they can collect from individual users in practice. In this paper, we propose a reinforcement learning algorithm to address this by training an interface to map raw command signals to actions using a combination of offline pre-training and online fine-tuning. To address the challenges posed by noisy command signals and sparse rewards, we develop a novel method for representing and inferring the user's long-term intent for a given trajectory. We primarily evaluate our method's ability to assist users who can only communicate through noisy, high-dimensional input channels through a user study in which 12 participants performed a simulated navigation task by using their eye gaze to modulate a 128-dimensional command signal from their webcam. The results show that our method enables successful goal navigation more often than a baseline directional interface, by learning to denoise user commands signals and provide shared autonomy assistance. We further evaluate on a simulated Sawyer pushing task with eye gaze control, and the Lunar Lander game with simulated user commands, and find that our method improves over baseline interfaces in these domains as well. Extensive ablation experiments with simulated user commands empirically motivate each component of our method.


page 1

page 7

page 11


ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement Learning

Building assistive interfaces for controlling robots through arbitrary, ...

First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximization

How can we train an assistive human-machine interface (e.g., an electrom...

X2T: Training an X-to-Text Typing Interface with Online Learning from User Feedback

We aim to help users communicate their intent to machines using flexible...

LIMIT: Learning Interfaces to Maximize Information Transfer

Robots can use auditory, visual, or haptic interfaces to convey informat...

Toward a Framework for Adaptive Impedance Control of an Upper-limb Prosthesis

This paper describes a novel framework for a human-machine interface tha...

Modeling online adaptive navigation in virtual environments based on PID control

It is well known that locomotion-dominated navigation tasks may highly p...

Assisted Perception: Optimizing Observations to Communicate State

We aim to help users estimate the state of the world in tasks like robot...

Please sign up or login with your details

Forgot password? Click here to reset