LEO: Learning Energy-based Models in Graph Optimization

by   Paloma Sodhi, et al.

We address the problem of learning observation models end-to-end for estimation. Robots operating in partially observable environments must infer latent states from multiple sensory inputs using observation models that capture the joint distribution between latent states and observations. This inference problem can be formulated as an objective over a graph that optimizes for the most likely sequence of states using all previous measurements. Prior work uses observation models that are either known a-priori or trained on surrogate losses independent of the graph optimizer. In this paper, we propose a method to directly optimize end-to-end tracking performance by learning observation models with the graph optimizer in the loop. This direct approach may appear, however, to require the inference algorithm to be fully differentiable, which many state-of-the-art graph optimizers are not. Our key insight is to instead formulate the problem as that of energy-based learning. We propose a novel approach, LEO, for learning observation models end-to-end with non-differentiable graph optimizers. LEO alternates between sampling trajectories from the graph posterior and updating the model to match these samples to ground truth trajectories. We propose a way to generate such samples efficiently using incremental Gauss-Newton solvers. We compare LEO against baselines on datasets drawn from two distinct tasks: navigation and real-world planar pushing. We show that LEO is able to learn complex observation models with lower errors and fewer samples. Supplementary video: https://youtu.be/qWcH9CBXs5c


page 2

page 7

page 12

page 13


Learning Observation Models with Incremental Non-Differentiable Graph Optimizers in the Loop for Robotics State Estimation

We consider the problem of learning observation models for robot state e...

STEADY: Simultaneous State Estimation and Dynamics Learning from Indirect Observations

Accurate kinodynamic models play a crucial role in many robotics applica...

Learning Tactile Models for Factor Graph-based State Estimation

We address the problem of estimating object pose from touch during manip...

Differentiable Factor Graph Optimization for Learning Smoothers

A recent line of work has shown that end-to-end optimization of Bayesian...

Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision

Denoising diffusion models are a powerful type of generative models used...

Shadows Shed Light on 3D Objects

3D reconstruction is a fundamental problem in computer vision, and the t...

A posteriori learning for quasi-geostrophic turbulence parametrization

The use of machine learning to build subgrid parametrizations for climat...

Please sign up or login with your details

Forgot password? Click here to reset