Semi-supervised Learning From Demonstration Through Program Synthesis: An Inspection Robot Case Study

by   Simón C. Smith, et al.

Semi-supervised learning improves the performance of supervised machine learning by leveraging methods from unsupervised learning to extract information not explicitly available in the labels. Through the design of a system that enables a robot to learn inspection strategies from a human operator, we present a hybrid semi-supervised system capable of learning interpretable and verifiable models from demonstrations. The system induces a controller program by learning from immersive demonstrations using sequential importance sampling. These visual servo controllers are parametrised by proportional gains and are visually verifiable through observation of the position of the robot in the environment. Clustering and effective particle size filtering allows the system to discover goals in the state space. These goals are used to label the original demonstration for end-to-end learning of behavioural models. The behavioural models are used for autonomous model predictive control and scrutinised for explanations. We implement causal sensitivity analysis to identify salient objects and generate counterfactual conditional explanations. These features enable decision making interpretation and post hoc discovery of the causes of a failure. The proposed system expands on previous approaches to program synthesis by incorporating repellers in the attribution prior of the sampling process. We successfully learn the hybrid system from an inspection scenario where an unmanned ground vehicle has to inspect, in a specific order, different areas of the environment. The system induces an interpretable computer program of the demonstration that can be synthesised to produce novel inspection behaviours. Importantly, the robot successfully runs the synthesised program on an unseen configuration of the environment while presenting explanations of its autonomous behaviour.


page 3

page 7

page 9

page 12

page 14

page 15

page 21


From explanation to synthesis: Compositional program induction for learning from demonstration

Hybrid systems are a compact and natural mechanism with which to address...

Program Synthesis for Robot Learning from Demonstrations

This paper presents a new synthesis-based approach for solving the Learn...

A Framework for Learning from Demonstration with Minimal Human Effort

We consider robot learning in the context of shared autonomy, where cont...

PLUNDER: Probabilistic Program Synthesis for Learning from Unlabeled and Noisy Demonstrations

Learning from demonstration (LfD) is a widely researched paradigm for te...

Few-Shot Learning of Force-Based Motions From Demonstration Through Pre-training of Haptic Representation

In many contact-rich tasks, force sensing plays an essential role in ada...

Prediction-Constrained Training for Semi-Supervised Mixture and Topic Models

Supervisory signals have the potential to make low-dimensional data repr...

Demonstration of CORNET: A System For Learning Spreadsheet Formatting Rules By Example

Data management and analysis tasks are often carried out using spreadshe...

Please sign up or login with your details

Forgot password? Click here to reset