Using Scene Graph Context to Improve Image Generation

by   Subarna Tripathi, et al.

Generating realistic images from scene graphs asks neural networks to be able to reason about object relationships and compositionality. As a relatively new task, how to properly ensure the generated images comply with scene graphs or how to measure task performance remains an open question. In this paper, we propose to harness scene graph context to improve image generation from scene graphs. We introduce a scene graph context network that pools features generated by a graph convolutional neural network that are then provided to both the image generation network and the adversarial loss. With the context network, our model is trained to not only generate realistic looking images, but also to better preserve non-spatial object relationships. We also define two novel evaluation metrics, the relation score and the mean opinion relation score, for this task that directly evaluate scene graph compliance. We use both quantitative and qualitative studies to demonstrate that our pro-posed model outperforms the state-of-the-art on this challenging task.


page 1

page 4

page 6

page 7

page 8

page 11


Image Generation from Freehand Scene Sketches

We introduce the first method for automatic image generation from scene-...

Interactive Image Generation Using Scene Graphs

Recent years have witnessed some exciting developments in the domain of ...

Scene Graph to Image Generation with Contextualized Object Layout Refinement

Generating high-quality images from scene graphs, that is, graphs that d...

Depth Structure Preserving Scene Image Generation

Key to automatically generate natural scene images is to properly arrang...

SGDraw: Scene Graph Drawing Interface Using Object-Oriented Representation

Scene understanding is an essential and challenging task in computer vis...

Compact Scene Graphs for Layout Composition and Patch Retrieval

Structured representations such as scene graphs serve as an efficient an...

Learning to Evaluate the Artness of AI-generated Images

Assessing the artness of AI-generated images continues to be a challenge...

Please sign up or login with your details

Forgot password? Click here to reset