How To Guide Your Learner: Imitation Learning with Active Adaptive Expert Involvement

by   Xu-Hui Liu, et al.
Nanjing University
The University of Hong Kong

Imitation learning aims to mimic the behavior of experts without explicit reward signals. Passive imitation learning methods which use static expert datasets typically suffer from compounding error, low sample efficiency, and high hyper-parameter sensitivity. In contrast, active imitation learning methods solicit expert interventions to address the limitations. However, recent active imitation learning methods are designed based on human intuitions or empirical experience without theoretical guarantee. In this paper, we propose a novel active imitation learning framework based on a teacher-student interaction model, in which the teacher's goal is to identify the best teaching behavior and actively affect the student's learning process. By solving the optimization objective of this framework, we propose a practical implementation, naming it AdapMen. Theoretical analysis shows that AdapMen can improve the error bound and avoid compounding error under mild conditions. Experiments on the MetaDrive benchmark and Atari 2600 games validate our theoretical analysis and show that our method achieves near-expert performance with much less expert involvement and total sampling steps than previous methods. The code is available at


page 10

page 12

page 13


Support-weighted Adversarial Imitation Learning

Adversarial Imitation Learning (AIL) is a broad family of imitation lear...

The Past and Present of Imitation Learning: A Citation Chain Study

Imitation Learning is a promising area of active research. Over the last...

Active Policy Improvement from Multiple Black-box Oracles

Reinforcement learning (RL) has made significant strides in various comp...

Active Imitation Learning from Multiple Non-Deterministic Teachers: Formulation, Challenges, and Algorithms

We formulate the problem of learning to imitate multiple, non-determinis...

Theoretical Analysis of Offline Imitation With Supplementary Dataset

Behavioral cloning (BC) can recover a good policy from abundant expert d...

Of Moments and Matching: Trade-offs and Treatments in Imitation Learning

We provide a unifying view of a large family of previous imitation learn...

Regression via Kirszbraun Extension with Applications to Imitation Learning

Learning by demonstration is a versatile and rapid mechanism for transfe...

Please sign up or login with your details

Forgot password? Click here to reset