Layout-Graph Reasoning for Fashion Landmark Detection

by   Weijiang Yu, et al.

Detecting dense landmarks for diverse clothes, as a fundamental technique for clothes analysis, has attracted increasing research attention due to its huge application potential. However, due to the lack of modeling underlying semantic layout constraints among landmarks, prior works often detect ambiguous and structure-inconsistent landmarks of multiple overlapped clothes in one person. In this paper, we propose to seamlessly enforce structural layout relationships among landmarks on the intermediate representations via multiple stacked layout-graph reasoning layers. We define the layout-graph as a hierarchical structure including a root node, body-part nodes (e.g. upper body, lower body), coarse clothes-part nodes (e.g. collar, sleeve) and leaf landmark nodes (e.g. left-collar, right-collar). Each Layout-Graph Reasoning(LGR) layer aims to map feature representations into structural graph nodes via a Map-to-Node module, performs reasoning over structural graph nodes to achieve global layout coherency via a layout-graph reasoning module, and then maps graph nodes back to enhance feature representations via a Node-to-Map module. The layout-graph reasoning module integrates a graph clustering operation to generate representations of intermediate nodes (bottom-up inference) and then a graph deconvolution operation (top-down inference) over the whole graph. Extensive experiments on two public fashion landmark datasets demonstrate the superiority of our model. Furthermore, to advance the fine-grained fashion landmark research for supporting more comprehensive clothes generation and attribute recognition, we contribute the first Fine-grained Fashion Landmark Dataset (FFLD) containing 200k images annotated with at most 32 key-points for 13 clothes types.


page 1

page 4

page 7


A Global-Local Emebdding Module for Fashion Landmark Detection

Detecting fashion landmarks is a fundamental technique for visual clothi...

Fashion Landmark Detection in the Wild

Visual fashion analysis has attracted many attentions in the recent year...

Recapture as You Want

With the increasing prevalence and more powerful camera systems of mobil...

Graph Transformer GANs for Graph-Constrained House Generation

We present a novel graph Transformer generative adversarial network (GTG...

graphTPP: A multivariate based method for interactive graph layout and analysis

Graph layout is the process of creating a visual representation of a gra...

Unconstrained Fashion Landmark Detection via Hierarchical Recurrent Transformer Networks

Fashion landmarks are functional key points defined on clothes, such as ...

Incorporating Structural Alternatives and Sharing into Hierarchy for Multiclass Object Recognition and Detection

This paper proposes a reconfigurable model to recognize and detect multi...

Please sign up or login with your details

Forgot password? Click here to reset