Customizing First Person Image Through Desired Actions

by   Shan Su, et al.

This paper studies a problem of inverse visual path planning: creating a visual scene from a first person action. Our conjecture is that the spatial arrangement of a first person visual scene is deployed to afford an action, and therefore, the action can be inversely used to synthesize a new scene such that the action is feasible. As a proof-of-concept, we focus on linking visual experiences induced by walking. A key innovation of this paper is a concept of ActionTunnel---a 3D virtual tunnel along the future trajectory encoding what the wearer will visually experience as moving into the scene. This connects two distinctive first person images through similar walking paths. Our method takes a first person image with a user defined future trajectory and outputs a new image that can afford the future motion. The image is created by combining present and future ActionTunnels in 3D where the missing pixels in adjoining area are computed by a generative adversarial network. Our work can provide a travel across different first person experiences in diverse real world scenes.


page 1

page 3

page 4

page 5

page 6

page 7

page 8


Synthesizing Images of Humans in Unseen Poses

We address the computational problem of novel human pose synthesis. Give...

VITON-GAN: Virtual Try-on Image Generator Trained with Adversarial Loss

Generating a virtual try-on image from in-shop clothing images and a mod...

Scene Aware Person Image Generation through Global Contextual Conditioning

Person image generation is an intriguing yet challenging problem. Howeve...

Predicting 3D Human Dynamics from Video

Given a video of a person in action, we can easily guess the 3D future m...

Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image

In this paper, we propose a deep convolutional recurrent neural network ...

Generating Continual Human Motion in Diverse 3D Scenes

We introduce a method to synthesize animator guided human motion across ...

Knowledge Transfer for Scene-specific Motion Prediction

When given a single frame of the video, humans can not only interpret th...

Please sign up or login with your details

Forgot password? Click here to reset