Learning Object Arrangements in 3D Scenes using Human Context

06/27/2012
by   Yun Jiang, et al.
0

We consider the problem of learning object arrangements in a 3D scene. The key idea here is to learn how objects relate to human poses based on their affordances, ease of use and reachability. In contrast to modeling object-object relationships, modeling human-object relationships scales linearly in the number of objects. We design appropriate density functions based on 3D spatial features to capture this. We learn the distribution of human poses in a scene using a variant of the Dirichlet process mixture model that allows sharing of the density function parameters across the same object types. Then we can reason about arrangements of the objects in the room based on these meaningful human poses. In our extensive experiments on 20 different rooms with a total of 47 objects, our algorithm predicted correct placements with an average error of 1.6 meters from ground truth. In arranging five real scenes, it received a score of 4.3/5 compared to 3.7 for the best baseline method.

READ FULL TEXT

page 1

page 6

page 7

research
12/01/2021

Pose2Room: Understanding 3D Scenes from Human Activities

With wearable IMU sensors, one can estimate human poses from wearable de...
research
11/29/2017

A Generative Model of 3D Object Layouts in Apartments

Understanding indoor scenes is an important task in computer vision. Thi...
research
01/23/2023

LEGO-Net: Learning Regular Rearrangements of Objects in Rooms

Humans universally dislike the task of cleaning up a messy room. If mach...
research
10/24/2017

Improving 6D Pose Estimation of Objects in Clutter via Physics-aware Monte Carlo Tree Search

This work proposes a process for efficiently searching over combinations...
research
01/08/2020

SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition

The ability to decompose complex multi-object scenes into meaningful abs...
research
08/01/2016

Modeling Context Between Objects for Referring Expression Understanding

Referring expressions usually describe an object using properties of the...
research
01/25/2023

Implicit Shape Model Trees: Recognition of 3-D Indoor Scenes and Prediction of Object Poses for Mobile Robots

For a mobile robot, we present an approach to recognize scenes in arrang...

Please sign up or login with your details

Forgot password? Click here to reset