OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation

by   Tong Wu, et al.
SenseTime Corporation
Nanyang Technological University
The Chinese University of Hong Kong
NetEase, Inc
The Hong Kong University of Science and Technology

Recent advances in modeling 3D objects mostly rely on synthetic datasets due to the lack of large-scale realscanned 3D databases. To facilitate the development of 3D perception, reconstruction, and generation in the real world, we propose OmniObject3D, a large vocabulary 3D object dataset with massive high-quality real-scanned 3D objects. OmniObject3D has several appealing properties: 1) Large Vocabulary: It comprises 6,000 scanned objects in 190 daily categories, sharing common classes with popular 2D datasets (e.g., ImageNet and LVIS), benefiting the pursuit of generalizable 3D representations. 2) Rich Annotations: Each 3D object is captured with both 2D and 3D sensors, providing textured meshes, point clouds, multiview rendered images, and multiple real-captured videos. 3) Realistic Scans: The professional scanners support highquality object scans with precise shapes and realistic appearances. With the vast exploration space offered by OmniObject3D, we carefully set up four evaluation tracks: a) robust 3D perception, b) novel-view synthesis, c) neural surface reconstruction, and d) 3D object generation. Extensive studies are performed on these four benchmarks, revealing new observations, challenges, and opportunities for future research in realistic 3D vision.


page 1

page 3

page 8

page 10

page 12

page 14

page 15

page 16


V3Det: Vast Vocabulary Visual Detection Dataset

Recent advances in detecting arbitrary objects in the real world are tra...

MVImgNet: A Large-scale Dataset of Multi-view Images

Being data-driven is one of the most iconic properties of deep learning ...

TAO: A Large-Scale Benchmark for Tracking Any Object

For many years, multi-object tracking benchmarks have focused on a handf...

A Real2Sim2Real Method for Robust Object Grasping with Neural Surface Reconstruction

Recent 3D-based manipulation methods either directly predict the grasp p...

SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling

Synthetic data has emerged as a promising source for 3D human research a...

Generating Datasets of 3D Garments with Sewing Patterns

Garments are ubiquitous in both real and many of the virtual worlds. The...

Objaverse-XL: A Universe of 10M+ 3D Objects

Natural language processing and 2D vision models have attained remarkabl...

Please sign up or login with your details

Forgot password? Click here to reset