research
∙
05/16/2023
Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation
Text-guided human motion generation has drawn significant interest becau...
share
research
∙
04/14/2023
Text-Conditional Contextualized Avatars For Zero-Shot Personalization
Recent large-scale text-to-image generation models have made significant...
research
∙
11/25/2022
SpaText: Spatio-Textual Representation for Controllable Image Generation
Recent text-to-image diffusion models are able to generate convincing re...
research
∙
05/03/2022
Episodic Memory Question Answering
Egocentric augmented reality devices such as wearable glasses passively ...
research
∙
04/07/2022
Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer
Videos are created to express emotion, exchange information, and share e...
research
∙
03/24/2022
Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors
Recent text-to-image generation methods provide a simple yet exciting co...
research
∙
10/27/2021
Telling Creative Stories Using Generative Visual Aids
Can visual artworks created using generative visual algorithms inspire h...
research
∙
07/13/2021
Dance2Music: Automatic Dance-driven Music Generation
Dance and music typically go hand in hand. The complexities in dance, mu...
research
∙
06/27/2021
Visual Conceptual Blending with Large-scale Language and Vision Models
We ask the question: to what extent can recent large-scale language and ...
research
∙
06/25/2021
Building Bridges: Generative Artworks to Explore AI Ethics
In recent years, there has been an increased emphasis on understanding a...
research
∙
06/04/2021
Human-Adversarial Visual Question Answering
Performance on the most commonly used Visual Question Answering dataset ...
research
∙
03/02/2021
ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations
With massive amounts of atomic simulation data available, there is a hug...
research
∙
01/28/2021
VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs
We present Vx2Text, a framework for text generation from multimodal inpu...
research
∙
12/21/2020
Object-Centric Diagnosis of Visual Reasoning
When answering questions about an image, it not only needs knowing what ...
research
∙
11/19/2020
Creative Sketch Generation
Sketching or doodling is a popular creative activity that people engage ...
research
∙
11/16/2020
Where Are You? Localization from Embodied Dialog
We present Where Are You? (WAY), a dataset of  6k dialogs in which two h...
research
∙
11/07/2020
Sim-to-Real Transfer for Vision-and-Language Navigation
We study the challenging problem of releasing a robot in a previously un...
research
∙
10/20/2020
SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency
Recent research in Visual Question Answering (VQA) has revealed state-of...
research
∙
10/20/2020
The Open Catalyst 2020 (OC20) Dataset and Community Challenges
Catalyst discovery and optimization is key to solving many societal and ...
research
∙
10/14/2020
An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storage
Scalable and cost-effective solutions to renewable energy storage are es...
research
∙
10/13/2020
Contrast and Classify: Alternate Training for Robust VQA
Recent Visual Question Answering (VQA) models have shown impressive perf...
research
∙
09/07/2020
Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agents
Recent work has presented embodied agents that can navigate to point-goa...
research
∙
07/24/2020
Dialog without Dialog Data: Learning Visual Dialog Agents from VQA Data
Can we develop visually grounded dialog agents that can efficiently adap...
research
∙
07/23/2020
Spatially Aware Multimodal Transformers for TextVQA
Textual cues are essential for everyday tasks like buying groceries and ...
research
∙
07/20/2020
Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation
We introduce a learning-based approach for room navigation using semanti...
research
∙
07/04/2020
Neuro-Symbolic Generative Art: A Preliminary Study
There are two classes of generative art approaches: neural, where a deep...
research
∙
06/21/2020
Feel The Music: Automatically Generating A Dance For An Input Song
We present a general computational approach that enables a machine to ge...
research
∙
05/15/2020
Exploring Crowd Co-creation Scenarios for Sketches
As a first step towards studying the ability of human crowds and machine...
research
∙
04/30/2020
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web
Following a navigation instruction such as 'Walk down the stairs and sto...
research
∙
04/19/2020
Are we pretraining it right? Digging deeper into visio-linguistic pretraining
Numerous recent works have proposed pretraining generic visio-linguistic...
research
∙
03/03/2020
Predicting A Creator's Preferences In, and From, Interactive Generative Art
As a lay user creates an art piece using an interactive generative art t...
research
∙
01/20/2020
SQuINTing at VQA Models: Interrogating VQA Models with Sub-Questions
Existing VQA datasets contain questions with varying levels of complexit...
research
∙
12/05/2019
Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline
Prior work in visual dialog has focused on training deep neural models o...
research
∙
12/05/2019
12-in-1: Multi-Task Vision and Language Representation Learning
Much of vision-and-language research focuses on a small but diverse set ...
research
∙
11/01/2019
Decentralized Distributed PPO: Solving PointGoal Navigation
We present Decentralized Distributed Proximal Policy Optimization (DD-PP...
research
∙
09/23/2019
Improving Generative Visual Dialog by Answering Diverse Questions
Prior work on training generative Visual Dialog models with reinforcemen...
research
∙
08/06/2019
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
We present ViLBERT (short for Vision-and-Language BERT), a model for lea...
research
∙
07/24/2019
Unsupervised Discovery of Decision States for Transfer in Reinforcement Learning
We present a hierarchical reinforcement learning (HRL) or options framew...
research
∙
07/03/2019
Chasing Ghosts: Instruction Following as Bayesian State Tracking
A visually-grounded navigation instruction can be interpreted as a seque...
research
∙
06/24/2019
RUBi: Reducing Unimodal Biases in Visual Question Answering
Visual Question Answering (VQA) is the task of answering questions about...
research
∙
05/18/2019
SplitNet: Sim2Sim and Task2Task Transfer for Embodied Visual Navigation
We propose SplitNet, a method for decoupling visual perception and polic...
research
∙
04/19/2019
Fashion++: Minimal Edits for Outfit Improvement
Given an outfit, what small changes would most improve its fashionabilit...
research
∙
04/19/2019
Emergence of Compositional Language with Deep Generational Transmission
Consider a collaborative task that requires communication. Two agents ar...
research
∙
04/18/2019
Towards VQA Models that can Read
Studies have shown that a dominant class of questions asked by visually ...
research
∙
04/16/2019
Counterfactual Visual Explanations
A counterfactual query is typically of the form 'For situation X, why wa...
research
∙
04/09/2019
Embodied Visual Recognition
Passive visual systems typically fail to recognize objects in the amodal...
research
∙
04/06/2019
Embodied Question Answering in Photorealistic Environments with Point Cloud Perception
To help bridge the gap between internet vision-style problems and the go...
research
∙
04/02/2019
Habitat: A Platform for Embodied AI Research
We present Habitat, a new platform for research in embodied artificial i...
research
∙
03/27/2019
Align2Ground: Weakly Supervised Phrase Grounding Guided by Image-Caption Alignment
We address the problem of grounding free-form textual phrases by using w...
research
∙
03/19/2019