Universal Morphology Control via Contextual Modulation

by   Zheng Xiong, et al.

Learning a universal policy across different robot morphologies can significantly improve learning efficiency and generalization in continuous control. However, it poses a challenging multi-task reinforcement learning problem, as the optimal policy may be quite different across robots and critically depend on the morphology. Existing methods utilize graph neural networks or transformers to handle heterogeneous state and action spaces across different morphologies, but pay little attention to the dependency of a robot's control policy on its morphology context. In this paper, we propose a hierarchical architecture to better model this dependency via contextual modulation, which includes two key submodules: (1) Instead of enforcing hard parameter sharing across robots, we use hypernetworks to generate morphology-dependent control parameters; (2) We propose a morphology-dependent attention mechanism to modulate the interactions between different limbs in a robot. Experimental results show that our method not only improves learning performance on a diverse set of training robots, but also generalizes better to unseen morphologies in a zero-shot fashion.


page 6

page 7


AnyMorph: Learning Transferable Polices By Inferring Agent Morphology

The prototypical approach to reinforcement learning involves training po...

Multi-embodiment Legged Robot Control as a Sequence Modeling Problem

Robots are traditionally bounded by a fixed embodiment during their oper...

A System for Morphology-Task Generalization via Unified Representation and Behavior Distillation

The rise of generalist large-scale models in natural language and vision...

REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy Transfer

A popular paradigm in robotic learning is to train a policy from scratch...

C^2:Co-design of Robots via Concurrent Networks Coupling Online and Offline Reinforcement Learning

With the rise of computing power, using data-driven approaches for co-de...

Controlling Robot Morphology from Incomplete Measurements

Mobile robots with complex morphology are essential for traversing rough...

DMAP: a Distributed Morphological Attention Policy for Learning to Locomote with a Changing Body

Biological and artificial agents need to deal with constant changes in t...

Please sign up or login with your details

Forgot password? Click here to reset