research
∙
05/24/2023
AMELI: Enhancing Multimodal Entity Linking with Fine-Grained Attributes
We propose attribute-aware multimodal entity linking, where the input is...
share
research
∙
03/31/2023
Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations
The abundance of instructional videos and their narrations over the Inte...
research
∙
03/23/2023
Learning and Verification of Task Structure in Instructional Videos
Given the enormous number of instructional videos available online, lear...
research
∙
03/04/2023
FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks
In the fashion domain, there exists a variety of vision-and-language (V+...
research
∙
02/28/2023
RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Data
Semi-supervised learning aims to train a model using limited labels. Sta...
research
∙
02/21/2023
Que2Engage: Embedding-based Retrieval for Relevant and Engaging Products at Facebook Marketplace
Embedding-based Retrieval (EBR) in e-commerce search is a powerful searc...
research
∙
01/05/2023
CiT: Curation in Training for Effective Vision-Language Data
Large vision-language models are generally applicable to many downstream...
research
∙
11/23/2022
Tell Me What Happened: Unifying Text-guided Video Completion via Multimodal Masked Video Generation
Generating a video given the first several static frames is challenging ...
research
∙
10/26/2022
FaD-VLP: Fashion Vision-and-Language Pre-training towards Unified Retrieval and Captioning
Multimodal tasks in the fashion domain have significant potential for e-...
research
∙
07/17/2022
FashionViL: Fashion-Focused Vision-and-Language Representation Learning
Large-scale Vision-and-Language (V+L) pre-training for representation le...
research
∙
04/01/2022
Generic Event Boundary Captioning: A Benchmark for Status Changes Understanding
Cognitive science has shown that humans perceive videos in terms of even...
research
∙
03/10/2022
LoopITR: Combining Dual and Cross Encoder Architectures for Image-Text Retrieval
Dual encoders and cross encoders have been widely used for image-text re...
research
∙
03/01/2022
Unsupervised Vision-and-Language Pre-training via Retrieval-based Multi-Granular Alignment
Vision-and-Language (V+L) pre-training models have achieved tremendous s...
research
∙
02/15/2022
CommerceMM: Large-Scale Commerce MultiModal Representation Learning with Omni Retrieval
We introduce CommerceMM - a multimodal model capable of providing a dive...
research
∙
06/08/2021
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
Most existing video-and-language (VidL) research focuses on a single dat...
research
∙
05/12/2021
Connecting What to Say With Where to Look by Modeling Human Attention Traces
We introduce a unified framework to jointly model images, text, and huma...
research
∙
10/15/2020
What is More Likely to Happen Next? Video-and-Language Future Event Prediction
Given a video with aligned dialogue, people can often infer what is more...
research
∙
05/15/2020
Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models
Recent Transformer-based large-scale pre-trained models have revolutioni...
research
∙
05/01/2020
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
We present HERO, a Hierarchical EncodeR for Omni-representation learning...
research
∙
03/26/2020
BachGAN: High-Resolution Image Synthesis from Salient Object Layout
We propose a new task towards more practical application for image gener...
research
∙
03/25/2020
VIOLIN: A Large-Scale Dataset for Video-and-Language Inference
We introduce a new task, Video-and-Language Inference, for joint multimo...
research
∙
01/24/2020
TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval
We introduce a new multimodal retrieval task - TV show Retrieval (TVR), ...
research
∙
09/25/2019
UNITER: Learning UNiversal Image-TExt Representations
Joint image-text embedding is the bedrock for most Vision-and-Language (...
research
∙
04/25/2019
TVQA+: Spatio-Temporal Grounding for Video Question Answering
We present the task of Spatio-Temporal Video Question Answering, which r...
research
∙
04/09/2019
Multi-Target Embodied Question Answering
Embodied Question Answering (EQA) is a relatively new task where an agen...
research
∙
04/08/2019
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout
A grand goal in AI is to build a robot that can accurately navigate base...
research
∙
09/05/2018
TVQA: Localized, Compositional Video Question Answering
Recent years have witnessed an increasing interest in image-based questi...
research
∙
01/24/2018
MAttNet: Modular Attention Network for Referring Expression Comprehension
In this paper, we address referring expression comprehension: localizing...
research
∙
08/09/2017
Hierarchically-Attentive RNN for Album Summarization and Storytelling
We address the problem of end-to-end visual storytelling. Given a photo ...
research
∙
12/30/2016
A Joint Speaker-Listener-Reinforcer Model for Referring Expressions
Referring expressions are natural language constructions used to identif...
research
∙
08/03/2016
Detailed Garment Recovery from a Single-View Image
Most recent garment capturing techniques rely on acquiring multiple view...
research
∙
07/31/2016
Modeling Context in Referring Expressions
Humans refer to objects in their environments all the time, especially i...
research
∙
05/31/2015