Using Approximate Models in Robot Learning

by   Ali Lenjani, et al.

Trajectory following is one of the complicated control problems when its dynamics are nonlinear, stochastic and include a large number of parameters. The problem has significant difficulties including a large number of trials required for data collection and a massive volume of computations required to find a closed-loop controller for high dimensional and stochastic domains. For solving this type of problem, if we have an appropriate reward function and dynamics model; finding an optimal control policy is possible by using model-based reinforcement learning and optimal control algorithms. However, defining an accurate dynamics model is not possible for complicated problems. Pieter Abbeel and Andrew Ng recently presented an algorithm that requires only an approximate model and only a small number of real-life trials. This algorithm has broad applicability; however, there are some problems regarding the convergence of the algorithm. In this research, required modifications are presented that provide more powerful assurance for converging to optimal control policy. Also updated algorithm is implemented to evaluate the efficiency of the new algorithm by comparing the acquired results with human expert performance. We are using differential dynamic programming (DDP) as the locally trajectory optimizer, and a 2D dynamics and kinematics simulator is used to evaluate the accuracy of the presented algorithm.


Decoupled Data Based Approach for Learning to Control Nonlinear Dynamical Systems

This paper addresses the problem of learning the optimal control policy ...

Model-Free Adaptive Optimal Control of Sequential Manufacturing Processes using Reinforcement Learning

A self-learning optimal control algorithm for sequential manufacturing p...

Optimization Strategies for Real-Time Control of an Autonomous Melting Probe

We present an optimization-based approach for trajectory planning and co...

RLOC: Neurobiologically Inspired Hierarchical Reinforcement Learning Algorithm for Continuous Control of Nonlinear Dynamical Systems

Nonlinear optimal control problems are often solved with numerical metho...

Combining Gaussian processes and polynomial chaos expansions for stochastic nonlinear model predictive control

Model predictive control is an advanced control approach for multivariab...

Neural Optimal Control using Learned System Dynamics

We study the problem of generating control laws for systems with unknown...

Modeling and Simulation of a Point to Point Spherical Articulated Manipulator using Optimal Control

This paper aims to design an optimal stability controller for a point to...

Please sign up or login with your details

Forgot password? Click here to reset