Robust 3D Garment Digitization from Monocular 2D Images for 3D Virtual Try-On Systems

by   Sahib Majithia, et al.

In this paper, we develop a robust 3D garment digitization solution that can generalize well on real-world fashion catalog images with cloth texture occlusions and large body pose variations. We assumed fixed topology parametric template mesh models for known types of garments (e.g., T-shirts, Trousers) and perform mapping of high-quality texture from an input catalog image to UV map panels corresponding to the parametric mesh model of the garment. We achieve this by first predicting a sparse set of 2D landmarks on the boundary of the garments. Subsequently, we use these landmarks to perform Thin-Plate-Spline-based texture transfer on UV map panels. Subsequently, we employ a deep texture inpainting network to fill the large holes (due to view variations self-occlusions) in TPS output to generate consistent UV maps. Furthermore, to train the supervised deep networks for landmark prediction texture inpainting tasks, we generated a large set of synthetic data with varying texture and lighting imaged from various views with the human present in a wide variety of poses. Additionally, we manually annotated a small set of fashion catalog images crawled from online fashion e-commerce platforms to finetune. We conduct thorough empirical evaluations and show impressive qualitative results of our proposed 3D garment texture solution on fashion catalog images. Such 3D garment digitization helps us solve the challenging task of enabling 3D Virtual Try-on.


page 1

page 4

page 5

page 6

page 7

page 8


3DBooSTeR: 3D Body Shape and Texture Recovery

We propose 3DBooSTeR, a novel method to recover a textured 3D body mesh ...

Cloth2Tex: A Customized Cloth Texture Generation Pipeline for 3D Virtual Try-On

Fabricating and designing 3D garments has become extremely demanding wit...

A deep learning based interactive sketching system for fashion images design

In this work, we propose an interactive system to design diverse high-qu...

FashionTex: Controllable Virtual Try-on with Text and Texture

Virtual try-on attracts increasing research attention as a promising way...

A Self-Organizing Neural Scheme for Door Detection in Different Environments

Doors are important landmarks for indoor mobile robot navigation and als...

3D Virtual Garment Modeling from RGB Images

We present a novel approach that constructs 3D virtual garment models fr...

Vehicle Reconstruction and Texture Estimation Using Deep Implicit Semantic Template Mapping

We introduce VERTEX, an effective solution to recover 3D shape and intri...

Please sign up or login with your details

Forgot password? Click here to reset