research
∙
08/16/2023
Answering Ambiguous Questions with a Database of Questions, Answers, and Revisions
Many open-domain questions are under-specified and thus have multiple po...
share
research
∙
07/24/2023
Contrastive Example-Based Control
While many real-world problems that might benefit from reinforcement lea...
research
∙
07/24/2023
A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning
As with any machine learning problem with limited data, effective offlin...
research
∙
06/28/2023
MultiZoo MultiBench: A Standardized Toolkit for Multimodal Deep Learning
Learning multimodal representations involves integrating information fro...
research
∙
06/26/2023
Localized Text-to-Image Generation for Free via Cross Attention Control
Despite the tremendous success in text-to-image generative models, local...
research
∙
06/08/2023
Factorized Contrastive Learning: Going Beyond Multi-view Redundancy
In a wide range of multimodal tasks, contrastive learning has become a p...
research
∙
06/07/2023
Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applications
In many machine learning systems that jointly learn from multiple modali...
research
∙
06/07/2023
Multimodal Fusion Interactions: A Study of Human and Automatic Quantification
Multimodal fusion of multiple heterogeneous and interconnected signals i...
research
∙
06/06/2023
Stabilizing Contrastive RL: Techniques for Offline Goal Reaching
In the same way that the computer vision (CV) and natural language proce...
research
∙
05/26/2023
Generating Images with Multimodal Language Models
We propose a method to fuse frozen text-only large language models (LLMs...
research
∙
05/25/2023
Imitating Task and Motion Planning with Visuomotor Transformers
Imitation learning is a powerful tool for training robot manipulation po...
research
∙
05/24/2023
SPRING: GPT-4 Out-performs RL Algorithms by Studying Papers and Reasoning
Open-world survival games pose significant challenges for AI algorithms ...
research
∙
05/03/2023
Plan, Eliminate, and Track – Language Models are Good Teachers for Embodied Agents
Pre-trained large language models (LLMs) capture procedural knowledge ab...
research
∙
02/23/2023
Quantifying Modeling Feature Interactions: An Information Decomposition Framework
The recent explosion of interest in multimodal applications has resulted...
research
∙
01/31/2023
Grounding Language Models to Images for Multimodal Generation
We propose an efficient method to ground pretrained text-only language m...
research
∙
12/20/2022
Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment
Despite recent progress towards scaling up multimodal vision-language mo...
research
∙
12/09/2022
Object Goal Navigation with End-to-End Self-Supervision
A household robot should be able to navigate to target locations without...
research
∙
11/10/2022
Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control
Pretrained language models have demonstrated extraordinary capabilities ...
research
∙
09/25/2022
Paraphrasing Is All You Need for Novel Object Captioning
Novel object captioning (NOC) aims to describe images containing objects...
research
∙
09/18/2022
Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective
While reinforcement learning (RL) methods that learn an internal model o...
research
∙
07/10/2022
Scalable Privacy-enhanced Benchmark Graph Generative Model for Graph Convolutional Networks
A surge of interest in Graph Convolutional Networks (GCN) has produced t...
research
∙
06/30/2022
MultiViz: An Analysis Benchmark for Visualizing and Understanding Multimodal Models
The promise of multimodal models for real-world applications has inspire...
research
∙
06/21/2022
A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Search
Physically rearranging objects is an important capability for embodied a...
research
∙
06/15/2022
Contrastive Learning as Goal-Conditioned Reinforcement Learning
In reinforcement learning (RL), it is easier to solve a task if given a ...
research
∙
06/07/2022
Imitating Past Successes can be Very Suboptimal
Prior work has proposed a simple strategy for reinforcement learning (RL...
research
∙
05/25/2022
Reasoning over Logically Interacted Conditions for Question Answering
Some questions have multiple answers that are not equally correct, i.e. ...
research
∙
03/21/2022
PACS: A Dataset for Physical Audiovisual CommonSense Reasoning
In order for AI to be safely deployed in real-world scenarios such as ho...
research
∙
03/03/2022
Zero-shot Domain Adaptation of Heterogeneous Graphs via Knowledge Transfer Networks
How can we make predictions for nodes in a heterogeneous graph when an e...
research
∙
03/03/2022
DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations
The ability for a human to understand an Artificial Intelligence (AI) mo...
research
∙
03/02/2022
HighMMT: Towards Modality and Task Generalization for High-Modality Representation Learning
Learning multimodal representations involves discovering correspondences...
research
∙
02/14/2022
Learning Weakly-Supervised Contrastive Representations
We argue that a form of the valuable information provided by the auxilia...
research
∙
02/11/2022
Conditional Contrastive Learning with Kernel
Conditional contrastive learning frameworks consider the conditional sam...
research
∙
12/02/2021
SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistency
In this paper, we explore how we can build upon the data and models of I...
research
∙
10/28/2021
Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives
Despite the potential of reinforcement learning (RL) for building genera...
research
∙
10/22/2021
C-Planning: An Automatic Curriculum for Learning Goal-Reaching Tasks
Goal-conditioned reinforcement learning (RL) can solve tasks in a wide r...
research
∙
10/13/2021
ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers
We describe a Question Answering (QA) dataset that contains complex ques...
research
∙
10/12/2021
FILM: Following Instructions in Language with Modular Methods
Recent methods for embodied instruction following are typically trained ...
research
∙
10/11/2021
Recurrent Model-Free RL is a Strong Baseline for Many POMDPs
Many problems in RL, such as meta RL, robust RL, and generalization in R...
research
∙
10/06/2021
Mismatched No More: Joint Model-Policy Optimization for Model-Based RL
Many model-based reinforcement learning (RL) methods follow a similar te...
research
∙
10/06/2021
The Information Geometry of Unsupervised Reinforcement Learning
How can a reinforcement learning (RL) agent prepare to solve downstream ...
research
∙
09/27/2021
FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding
The few-shot natural language understanding (NLU) task has attracted muc...
research
∙
09/07/2021
Robust Predictable Control
Many of the challenges facing today's reinforcement learning (RL) algori...
research
∙
07/15/2021
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
Learning multimodal representations involves integrating information fro...
research
∙
06/24/2021
Towards Understanding and Mitigating Social Biases in Language Models
As machine learning methods are deployed in real-world settings such as ...
research
∙
06/24/2021
Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data
Mental health conditions remain underdiagnosed even in countries with co...
research
∙
06/14/2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Self-supervised approaches for speech representation learning are challe...
research
∙
06/14/2021
Online Sub-Sampling for Reinforcement Learning with General Function Approximation
Designing provably efficient algorithms with general function approximat...
research
∙
06/05/2021
Integrating Auxiliary Information in Self-supervised Learning
This paper presents to integrate the auxiliary information (e.g., additi...
research
∙
06/05/2021
Conditional Contrastive Learning: Removing Undesirable Information in Self-Supervised Representations
Self-supervised learning is a form of unsupervised learning that leverag...
research
∙
06/01/2021