b'Yuke Zhu'

research

∙ 09/05/2023

Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperation

We tackle the problem of developing humanoid loco-manipulation skills wi...

0 Mingyo Seo, et al. ∙

research

∙ 07/21/2023

Symbolic State Space Optimization for Long Horizon Mobile Manipulation Planning

In existing task and motion planning (TAMP) research, it is a common ass...

0 Xiaohan Zhang, et al. ∙

research

∙ 06/15/2023

Granger-Causal Hierarchical Skill Discovery

Reinforcement Learning (RL) has shown promising results learning policie...

0 Caleb Chuck, et al. ∙

research

∙ 06/05/2023

LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning

Lifelong learning offers a promising paradigm of building a generalist a...

0 Bo Liu, et al. ∙

research

∙ 05/25/2023

Voyager: An Open-Ended Embodied Agent with Large Language Models

We introduce Voyager, the first LLM-powered embodied lifelong learning a...

16 Guanzhi Wang, et al. ∙

research

∙ 05/22/2023

Fast Monocular Scene Reconstruction with Global-Sparse Local-Dense Grids

Indoor scene reconstruction from monocular images has long been sought a...

2 Wei Dong, et al. ∙

research

∙ 02/09/2023

Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning

Augmenting pretrained language models (LMs) with a vision encoder (e.g.,...

0 Zhuolin Yang, et al. ∙

research

∙ 11/15/2022

Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment

With the rapid growth of computing powers and recent advances in deep le...

0 Huihan Liu, et al. ∙

research

∙ 10/20/2022

Learning and Retrieval from Prior Data for Skill-based Imitation Learning

Imitation learning offers a promising path for robots to learn general-p...

2 Soroush Nasiriany, et al. ∙

research

∙ 10/20/2022

VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors

We introduce VIOLA, an object-centric imitation learning approach to lea...

0 Yifeng Zhu, et al. ∙

research

∙ 09/19/2022

Learning to Walk by Steering: Perceptive Quadrupedal Locomotion in Dynamic Environments

We tackle the problem of perceptive locomotion in dynamic environments. ...

0 Mingyo Seo, et al. ∙

research

∙ 06/27/2022

Causal Dynamics Learning for Task-Independent State Abstraction

Learning dynamics models accurately is an important goal for Model-Based...

14 Zizhao Wang, et al. ∙

research

∙ 06/17/2022

MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge

Autonomous agents have made great strides in specialist domains like Ata...

19 Linxi Fan, et al. ∙

research

∙ 05/27/2022

Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions

A significant gap remains between today's visual pattern recognition mod...

16 Huaizu Jiang, et al. ∙

research

∙ 05/04/2022

COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles

Optical sensors and learning algorithms for autonomous vehicles have dra...

6 Jiaxun Cui, et al. ∙

research

∙ 04/24/2022

RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning

Reasoning about visual relationships is central to how humans interpret ...

6 Xiaojian Ma, et al. ∙

research

∙ 02/22/2022

Visually Grounded Task and Motion Planning for Mobile Manipulation

Task and motion planning (TAMP) algorithms aim to help robots achieve ta...

0 Xiaohan Zhang, et al. ∙

research

∙ 02/03/2022

Pre-Trained Language Models for Interactive Decision-Making

Language model (LM) pre-training has proven useful for a wide variety of...

68 Shuang Li, et al. ∙

research

∙ 11/15/2021

Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularization

Skill chaining is a promising approach for synthesizing complex behavior...

9 Youngwoon Lee, et al. ∙

research

∙ 10/27/2021

Reinforcement Learning in Factored Action Spaces using Tensor Decompositions

We present an extended abstract for the previously published work TESSER...

4 Anuj Mahajan, et al. ∙

research

∙ 10/07/2021

Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks

Realistic manipulation tasks require a robot to interact with an environ...

3 Soroush Nasiriany, et al. ∙

research

∙ 10/02/2021

OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation

Learning performant robot manipulation policies can be challenging due t...

1 Josiah Wong, et al. ∙

research

∙ 09/28/2021

Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation

We tackle real-world long-horizon robot manipulation tasks through skill...

1 Yifeng Zhu, et al. ∙

research

∙ 07/15/2021

MultiBench: Multiscale Benchmarks for Multimodal Representation Learning

Learning multimodal representations involves integrating information fro...

0 Paul Pu Liang, et al. ∙

research

∙ 06/26/2021

Discovering Generalizable Skills via Automated Generation of Diverse Tasks

The learning efficiency and generalization ability of an intelligent age...

5 Kuan Fang, et al. ∙

research

∙ 06/17/2021

SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies

Generalization has been a long-standing challenge for reinforcement lear...

3 Linxi Fan, et al. ∙

research

∙ 05/31/2021

Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learning

Reinforcement Learning in large action spaces is a challenging problem. ...

33 Anuj Mahajan, et al. ∙

research

∙ 05/18/2021

Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition

In real-world multiagent systems, agents with different capabilities may...

1 Bo Liu, et al. ∙

research

∙ 05/13/2021

DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

We introduce DiscoBox, a novel framework that jointly learns instance se...

8 Shiyi Lan, et al. ∙

research

∙ 03/22/2021

Dynamic Metric Learning: Towards a Scalable Metric Space to Accommodate Multiple Semantic Scales

This paper introduces a new fundamental characteristic, , the dynamic ra...

0 Yifan Sun, et al. ∙

research

∙ 12/22/2020

Emergent Hand Morphology and Control from Optimizing Robust Grasps of Diverse Objects

Evolution in nature illustrates that the creatures' biological structure...

0 Xinlei Pan, et al. ∙

research

∙ 12/14/2020

Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphs

We present a visually grounded hierarchical planning algorithm for long-...

0 Yifeng Zhu, et al. ∙

research

∙ 12/12/2020

Learning Multi-Arm Manipulation Through Collaborative Teleoperation

Imitation Learning (IL) is a powerful paradigm to teach robots to perfor...

7 Albert Tung, et al. ∙

research

∙ 12/12/2020

Human-in-the-Loop Imitation Learning using Remote Teleoperation

Imitation Learning is a promising paradigm for learning complex robot ma...

15 Ajay Mandlekar, et al. ∙

research

∙ 12/01/2020

Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors

Using sensor data from multiple modalities presents an opportunity to en...

0 Michelle A. Lee, et al. ∙

research

∙ 11/17/2020

Deep Affordance Foresight: Planning Through What Can Be Done in the Future

Planning in realistic environments requires searching in large planning ...

0 Danfei Xu, et al. ∙

research

∙ 11/16/2020

Fast Uncertainty Quantification for Deep Object Pose Estimation

Deep learning-based object pose estimators are often unreliable and over...

0 Guanya Shi, et al. ∙

research

∙ 10/02/2020

Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning

Humans have an inherent ability to learn novel concepts from only a few ...

10 Weili Nie, et al. ∙

research

∙ 09/25/2020

robosuite: A Modular Simulation Framework and Benchmark for Robot Learning

robosuite is a simulation framework for robot learning powered by the Mu...

7 Yuke Zhu, et al. ∙

research

∙ 09/21/2020

Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion

We present a hierarchical framework that combines model-based control an...

0 Xingye Da, et al. ∙

research

∙ 08/17/2020

OCEAN: Online Task Inference for Compositional Tasks with Context Adaptation

Real-world tasks often exhibit a compositional structure that contains a...

1 Hongyu Ren, et al. ∙

research

∙ 08/04/2020

Spherical Feature Transform for Deep Metric Learning

Data augmentation in feature space is effective to increase data diversi...

0 Yuke Zhu, et al. ∙

research

∙ 07/01/2020

Adaptive Procedural Task Generation for Hard-Exploration Problems

We introduce Adaptive Procedural Task Generation (APT-Gen), an approach ...

18 Kuan Fang, et al. ∙

research

∙ 11/11/2019

Scaling Robot Supervision to Hundreds of Hours with RoboTurk: Robotic Manipulation Dataset through Human Reasoning and Dexterity

Large, richly annotated datasets have accelerated progress in fields suc...

0 Ajay Mandlekar, et al. ∙

research

∙ 10/29/2019

Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation

The fundamental challenge of planning for multi-step manipulation is to ...

10 Kuan Fang, et al. ∙

research

∙ 10/26/2019

KETO: Learning Keypoint Representations for Tool Manipulation

We aim to develop an algorithm for robots to manipulate novel objects as...

0 Zengyi Qin, et al. ∙

research

∙ 10/03/2019

Causal Induction from Visual Observations for Goal Directed Tasks

Causal reasoning has been an indispensable capability for humans and oth...

7 Suraj Nair, et al. ∙

research

∙ 09/28/2019

Regression Planning Networks

Recent learning-to-plan methods have shown promising results on planning...

87 Danfei Xu, et al. ∙

research

∙ 09/28/2019

Dual Sequential Monte Carlo: Tunneling Filtering and Planning in Continuous POMDPs

We present the DualSMC network that solves continuous POMDPs by learning...

15 Yunbo Wang, et al. ∙

research

∙ 09/27/2019

SURREAL-System: Fully-Integrated Stack for Distributed Deep Reinforcement Learning

We present an overview of SURREAL-System, a reproducible, flexible, and ...

0 Linxi Fan, et al. ∙

Yuke Zhu

Featured Co-authors