Unconditional Scene Graph Generation

by   Sarthak Garg, et al.

Despite recent advancements in single-domain or single-object image generation, it is still challenging to generate complex scenes containing diverse, multiple objects and their interactions. Scene graphs, composed of nodes as objects and directed-edges as relationships among objects, offer an alternative representation of a scene that is more semantically grounded than images. We hypothesize that a generative model for scene graphs might be able to learn the underlying semantic structure of real-world scenes more effectively than images, and hence, generate realistic novel scenes in the form of scene graphs. In this work, we explore a new task for the unconditional generation of semantic scene graphs. We develop a deep auto-regressive model called SceneGraphGen which can directly learn the probability distribution over labelled and directed graphs using a hierarchical recurrent architecture. The model takes a seed object as input and generates a scene graph in a sequence of steps, each step generating an object node, followed by a sequence of relationship edges connecting to the previous nodes. We show that the scene graphs generated by SceneGraphGen are diverse and follow the semantic patterns of real-world scenes. Additionally, we demonstrate the application of the generated graphs in image synthesis, anomaly detection and scene graph completion.


page 7

page 12

page 14


Learning Canonical Representations for Scene Graph to Image Generation

Generating realistic images of complex visual scenes becomes very challe...

Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions

Scene understanding has been of high interest in computer vision. It enc...

Transforming Image Generation from Scene Graphs

Generating images from semantic visual knowledge is a challenging task, ...

Compact Scene Graphs for Layout Composition and Patch Retrieval

Structured representations such as scene graphs serve as an efficient an...

Semantically Controllable Scene Generation with Guidance of Explicit Knowledge

Deep Generative Models (DGMs) are known for their superior capability in...

Generating unseen complex scenes: are we there yet?

Although recent complex scene conditional generation models generate inc...

Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs

Controllable scene synthesis consists of generating 3D information that ...

Please sign up or login with your details

Forgot password? Click here to reset