Semantic State Estimation in Cloth Manipulation Tasks

by   Georgies Tzelepis, et al.
Universitat Politècnica de Catalunya
Högskolan i Halmstad

Understanding of deformable object manipulations such as textiles is a challenge due to the complexity and high dimensionality of the problem. Particularly, the lack of a generic representation of semantic states (e.g., crumpled, diagonally folded) during a continuous manipulation process introduces an obstacle to identify the manipulation type. In this paper, we aim to solve the problem of semantic state estimation in cloth manipulation tasks. For this purpose, we introduce a new large-scale fully-annotated RGB image dataset showing various human demonstrations of different complicated cloth manipulations. We provide a set of baseline deep networks and benchmark them on the problem of semantic state estimation using our proposed dataset. Furthermore, we investigate the scalability of our semantic state estimation framework in robot monitoring tasks of long and complex cloth manipulations.


page 1

page 3

page 4

page 5

page 6


Learning Deformable Object Manipulation from Expert Demonstrations

We present a novel Learning from Demonstration (LfD) method, Deformable ...

Robust Shape Estimation for 3D Deformable Object Manipulation

Existing shape estimation methods for deformable object manipulation suf...

Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations

Dexterous multi-fingered hands are extremely versatile and provide a gen...

The dGLI Cloth Coordinates: A Topological Representation for Semantic Classification of Cloth States

Robotic manipulation of cloth is a highly complex task because of its in...

Scene Graph for Embodied Exploration in Cluttered Scenario

The ability to handle objects in cluttered environment has been long ant...

Embodied Referring Expression for Manipulation Question Answering in Interactive Environment

Embodied agents are expected to perform more complicated tasks in an int...

Semantic Decomposition and Recognition of Long and Complex Manipulation Action Sequences

Understanding continuous human actions is a non-trivial but important pr...

Code Repositories


A fully labelled dataset for Scemantic States of cloth manipulation tasks

view repo

Please sign up or login with your details

Forgot password? Click here to reset