Ruslan Salakhutdinov

research

∙ 08/16/2023

Answering Ambiguous Questions with a Database of Questions, Answers, and Revisions

Many open-domain questions are under-specified and thus have multiple po...

0 Haitian Sun, et al. ∙

research

∙ 07/24/2023

Contrastive Example-Based Control

While many real-world problems that might benefit from reinforcement lea...

0 Kyle Hatch, et al. ∙

research

∙ 07/24/2023

A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning

As with any machine learning problem with limited data, effective offlin...

0 Benjamin Eysenbach, et al. ∙

research

∙ 06/28/2023

MultiZoo MultiBench: A Standardized Toolkit for Multimodal Deep Learning

Learning multimodal representations involves integrating information fro...

0 Paul Pu Liang, et al. ∙

research

∙ 06/26/2023

Localized Text-to-Image Generation for Free via Cross Attention Control

Despite the tremendous success in text-to-image generative models, local...

0 Yutong He, et al. ∙

research

∙ 06/08/2023

Factorized Contrastive Learning: Going Beyond Multi-view Redundancy

In a wide range of multimodal tasks, contrastive learning has become a p...

0 Paul Pu Liang, et al. ∙

research

∙ 06/07/2023

Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applications

In many machine learning systems that jointly learn from multiple modali...

6 Paul Pu Liang, et al. ∙

research

∙ 06/07/2023

Multimodal Fusion Interactions: A Study of Human and Automatic Quantification

Multimodal fusion of multiple heterogeneous and interconnected signals i...

0 Paul Pu Liang, et al. ∙

research

∙ 06/06/2023

Stabilizing Contrastive RL: Techniques for Offline Goal Reaching

In the same way that the computer vision (CV) and natural language proce...

0 Chongyi Zheng, et al. ∙

research

∙ 05/26/2023

Generating Images with Multimodal Language Models

We propose a method to fuse frozen text-only large language models (LLMs...

0 Jing Yu Koh, et al. ∙

research

∙ 05/25/2023

Imitating Task and Motion Planning with Visuomotor Transformers

Imitation learning is a powerful tool for training robot manipulation po...

0 Murtaza Dalal, et al. ∙

research

∙ 05/24/2023

SPRING: GPT-4 Out-performs RL Algorithms by Studying Papers and Reasoning

Open-world survival games pose significant challenges for AI algorithms ...

0 Yue Wu, et al. ∙

research

∙ 05/03/2023

Plan, Eliminate, and Track – Language Models are Good Teachers for Embodied Agents

Pre-trained large language models (LLMs) capture procedural knowledge ab...

0 Yue Wu, et al. ∙

research

∙ 02/23/2023

Quantifying Modeling Feature Interactions: An Information Decomposition Framework

The recent explosion of interest in multimodal applications has resulted...

0 Paul Pu Liang, et al. ∙

research

∙ 01/31/2023

Grounding Language Models to Images for Multimodal Generation

We propose an efficient method to ground pretrained text-only language m...

0 Jing Yu Koh, et al. ∙

research

∙ 12/20/2022

Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment

Despite recent progress towards scaling up multimodal vision-language mo...

9 rohan-pandey, et al. ∙

research

∙ 12/09/2022

Object Goal Navigation with End-to-End Self-Supervision

A household robot should be able to navigate to target locations without...

0 So Yeon Min, et al. ∙

research

∙ 11/10/2022

Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control

Pretrained language models have demonstrated extraordinary capabilities ...

0 Xiang Fan, et al. ∙

research

∙ 09/25/2022

Paraphrasing Is All You Need for Novel Object Captioning

Novel object captioning (NOC) aims to describe images containing objects...

3 Cheng-Fu Yang, et al. ∙

research

∙ 09/18/2022

Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective

While reinforcement learning (RL) methods that learn an internal model o...

6 Raj Ghugare, et al. ∙

research

∙ 07/10/2022

Scalable Privacy-enhanced Benchmark Graph Generative Model for Graph Convolutional Networks

A surge of interest in Graph Convolutional Networks (GCN) has produced t...

0 Minji Yoon, et al. ∙

research

∙ 06/30/2022

MultiViz: An Analysis Benchmark for Visualizing and Understanding Multimodal Models

The promise of multimodal models for real-world applications has inspire...

2 Paul Pu Liang, et al. ∙

research

∙ 06/21/2022

A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Search

Physically rearranging objects is an important capability for embodied a...

0 Brandon Trabucco, et al. ∙

research

∙ 06/15/2022

Contrastive Learning as Goal-Conditioned Reinforcement Learning

In reinforcement learning (RL), it is easier to solve a task if given a ...

0 Benjamin Eysenbach, et al. ∙

research

∙ 06/07/2022

Imitating Past Successes can be Very Suboptimal

Prior work has proposed a simple strategy for reinforcement learning (RL...

0 Benjamin Eysenbach, et al. ∙

research

∙ 05/25/2022

Reasoning over Logically Interacted Conditions for Question Answering

Some questions have multiple answers that are not equally correct, i.e. ...

0 Haitian Sun, et al. ∙

research

∙ 03/21/2022

PACS: A Dataset for Physical Audiovisual CommonSense Reasoning

In order for AI to be safely deployed in real-world scenarios such as ho...

4 Samuel Yu, et al. ∙

research

∙ 03/03/2022

Zero-shot Domain Adaptation of Heterogeneous Graphs via Knowledge Transfer Networks

How can we make predictions for nodes in a heterogeneous graph when an e...

0 Minji Yoon, et al. ∙

research

∙ 03/03/2022

DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations

The ability for a human to understand an Artificial Intelligence (AI) mo...

8 Yiwei Lyu, et al. ∙

research

∙ 03/02/2022

HighMMT: Towards Modality and Task Generalization for High-Modality Representation Learning

Learning multimodal representations involves discovering correspondences...

10 Paul Pu Liang, et al. ∙

research

∙ 02/14/2022

Learning Weakly-Supervised Contrastive Representations

We argue that a form of the valuable information provided by the auxilia...

16 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 02/11/2022

Conditional Contrastive Learning with Kernel

Conditional contrastive learning frameworks consider the conditional sam...

10 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 12/02/2021

SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistency

In this paper, we explore how we can build upon the data and models of I...

9 Devendra Singh Chaplot, et al. ∙

research

∙ 10/28/2021

Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives

Despite the potential of reinforcement learning (RL) for building genera...

0 Murtaza Dalal, et al. ∙

research

∙ 10/22/2021

C-Planning: An Automatic Curriculum for Learning Goal-Reaching Tasks

Goal-conditioned reinforcement learning (RL) can solve tasks in a wide r...

0 Tianjun Zhang, et al. ∙

research

∙ 10/13/2021

ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers

We describe a Question Answering (QA) dataset that contains complex ques...

0 Haitian Sun, et al. ∙

research

∙ 10/12/2021

FILM: Following Instructions in Language with Modular Methods

Recent methods for embodied instruction following are typically trained ...

1 So Yeon Min, et al. ∙

research

∙ 10/11/2021

Recurrent Model-Free RL is a Strong Baseline for Many POMDPs

Many problems in RL, such as meta RL, robust RL, and generalization in R...

0 Tianwei Ni, et al. ∙

research

∙ 10/06/2021

Mismatched No More: Joint Model-Policy Optimization for Model-Based RL

Many model-based reinforcement learning (RL) methods follow a similar te...

4 Benjamin Eysenbach, et al. ∙

research

∙ 10/06/2021

The Information Geometry of Unsupervised Reinforcement Learning

How can a reinforcement learning (RL) agent prepare to solve downstream ...

0 Benjamin Eysenbach, et al. ∙

research

∙ 09/27/2021

FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding

The few-shot natural language understanding (NLU) task has attracted muc...

8 Yanan Zheng, et al. ∙

research

∙ 09/07/2021

Robust Predictable Control

Many of the challenges facing today's reinforcement learning (RL) algori...

0 Benjamin Eysenbach, et al. ∙

research

∙ 07/15/2021

MultiBench: Multiscale Benchmarks for Multimodal Representation Learning

Learning multimodal representations involves integrating information fro...

0 Paul Pu Liang, et al. ∙

research

∙ 06/24/2021

Towards Understanding and Mitigating Social Biases in Language Models

As machine learning methods are deployed in real-world settings such as ...

16 Paul Pu Liang, et al. ∙

research

∙ 06/24/2021

Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data

Mental health conditions remain underdiagnosed even in countries with co...

12 Paul Pu Liang, et al. ∙

research

∙ 06/14/2021

HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units

Self-supervised approaches for speech representation learning are challe...

0 Wei-Ning Hsu, et al. ∙

research

∙ 06/14/2021

Online Sub-Sampling for Reinforcement Learning with General Function Approximation

Designing provably efficient algorithms with general function approximat...

0 Dingwen Kong, et al. ∙

research

∙ 06/05/2021

Integrating Auxiliary Information in Self-supervised Learning

This paper presents to integrate the auxiliary information (e.g., additi...

2 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 06/05/2021

Conditional Contrastive Learning: Removing Undesirable Information in Self-Supervised Representations

Self-supervised learning is a form of unsupervised learning that leverag...

44 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 06/01/2021

End-to-End Multihop Retrieval for Compositional Question Answering over Long Documents

Answering complex questions from long documents requires aggregating mul...

0 Haitian Sun, et al. ∙

Ruslan Salakhutdinov

Featured Co-authors