b'Jianfeng Gao'

research

∙ 09/18/2023

Multimodal Foundation Models: From Specialists to General-Purpose Assistants

This paper presents a comprehensive survey of the taxonomy and evolution...

0 Chunyuan Li, et al. ∙

research

∙ 09/18/2023

MindAgent: Emergent Gaming Interaction

Large Language Models (LLMs) have the capacity of performing complex sch...

0 Ran Gong, et al. ∙

research

∙ 09/18/2023

An Empirical Study of Scaling Instruct-Tuned Large Multimodal Models

Visual instruction tuning has recently shown encouraging progress with o...

0 Yadong Lu, et al. ∙

research

∙ 06/16/2023

Demystifying GPT Self-Repair for Code Generation

Large Language Models (LLMs) have shown remarkable aptitude in code gene...

0 Theo X. Olausson, et al. ∙

research

∙ 06/12/2023

Augmenting Language Models with Long-Term Memory

Existing large language models (LLMs) can only afford fix-sized inputs d...

0 Weizhi Wang, et al. ∙

research

∙ 06/01/2023

LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

Conversational generative AI has demonstrated remarkable promise for emp...

1 Chunyuan Li, et al. ∙

research

∙ 06/01/2023

Differentiable Tree Operations Promote Compositional Generalization

In the context of structure-to-structure transformation tasks, learning ...

0 Paul Soulos, et al. ∙

research

∙ 05/23/2023

Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding

Scientific literature understanding tasks have gained significant attent...

0 Yu Zhang, et al. ∙

research

∙ 05/21/2023

Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers

This paper explores the effectiveness of model-generated signals in impr...

0 Linyuan Gong, et al. ∙

research

∙ 05/04/2023

Chain-of-Skills: A Configurable Model for Open-domain Question Answering

The retrieval model is an indispensable component for real-world knowled...

0 Kaixin Ma, et al. ∙

research

∙ 04/17/2023

Bridging Discrete and Backpropagation: Straight-Through and Beyond

Backpropagation, the cornerstone of deep learning, is limited to computi...

0 Liyuan Liu, et al. ∙

research

∙ 04/06/2023

Instruction Tuning with GPT-4

Prior work has shown that finetuning large language models (LLMs) using ...

0 Baolin Peng, et al. ∙

research

∙ 03/28/2023

Pre-training Transformers for Knowledge Graph Completion

Learning transferable representation of knowledge graphs (KGs) is challe...

0 Sanxing Chen, et al. ∙

research

∙ 03/02/2023

Interactive Text Generation

Users interact with text, image, code, or other editors on a daily basis...

5 Felix Faltings, et al. ∙

research

∙ 02/24/2023

Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback

Large language models (LLMs), such as ChatGPT, are able to generate huma...

0 Baolin Peng, et al. ∙

research

∙ 02/22/2023

Guiding Large Language Models via Directional Stimulus Prompting

We introduce a new framework, Directional Stimulus Prompting, that uses ...

0 Zekun Li, et al. ∙

research

∙ 01/17/2023

Learning Customized Visual Models with Retrieval-Augmented Knowledge

Image-text contrastive learning models such as CLIP have demonstrated st...

10 Haotian Liu, et al. ∙

research

∙ 01/17/2023

GLIGEN: Open-Set Grounded Text-to-Image Generation

Large-scale text-to-image diffusion models have made amazing advances. H...

1 Yuheng Li, et al. ∙

research

∙ 12/21/2022

Generalized Decoding for Pixel, Image, and Language

We present X-Decoder, a generalized decoding model that can predict pixe...

10 Xueyan Zou, et al. ∙

research

∙ 12/21/2022

Language Models as Inductive Reasoners

Inductive reasoning is a core component of human intelligence. In the pa...

0 Zonglin Yang, et al. ∙

research

∙ 12/20/2022

DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization

Dialogue summarization has recently garnered significant attention due t...

0 Yu Li, et al. ∙

research

∙ 12/20/2022

Enhancing Task Bot Engagement with Synthesized Open-Domain Dialog

Many efforts have been made to construct dialog systems for different ty...

0 Miaoran Li, et al. ∙

research

∙ 12/15/2022

Efficient Long Sequence Modeling via State Space Augmented Transformer

Transformer models have achieved superior performance in various natural...

0 Simiao Zuo, et al. ∙

research

∙ 12/04/2022

Grounded Keys-to-Text Generation: Towards Factual Open-Ended Generation

Large pre-trained language models have recently enabled open-ended gener...

0 Faeze Brahman, et al. ∙

research

∙ 11/30/2022

ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Format

Diverse data formats and ontologies of task-oriented dialogue (TOD) data...

0 Qi Zhu, et al. ∙

research

∙ 11/25/2022

CodeExp: Explanatory Code Document Generation

Developing models that can automatically generate detailed code explanat...

0 Haotian Cui, et al. ∙

research

∙ 11/17/2022

Execution-based Evaluation for Data Science Code Generation Models

Code generation models can benefit data scientists' productivity by auto...

0 Junjie Huang, et al. ∙

research

∙ 10/31/2022

AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning

Standard fine-tuning of large pre-trained language models (PLMs) for dow...

0 Yaqing Wang, et al. ∙

research

∙ 10/25/2022

Lafite2: Few-shot Text-to-Image Generation

Text-to-image generation models have progressed considerably in recent y...

0 Yufan Zhou, et al. ∙

research

∙ 10/22/2022

Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge

We propose a novel open-domain question answering (ODQA) framework for a...

0 Kaixin Ma, et al. ∙

research

∙ 10/17/2022

Vision-Language Pre-training: Basics, Recent Advances, and Future Trends

This paper surveys vision-language pre-training (VLP) methods for multim...

0 Zhe Gan, et al. ∙

research

∙ 10/14/2022

AutoMoE: Neural Architecture Search for Efficient Sparsely Activated Transformers

Neural architecture search (NAS) has demonstrated promising results on i...

1 Ganesh Jawahar, et al. ∙

research

∙ 10/11/2022

Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering

Given its effectiveness on knowledge-intensive natural language processi...

0 Hao Cheng, et al. ∙

research

∙ 09/07/2022

K-VIL: Keypoints-based Visual Imitation Learning

Visual imitation learning provides efficient and intuitive solutions for...

0 Jianfeng Gao, et al. ∙

research

∙ 08/30/2022

Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learning

We present an efficient bi-encoder framework for named entity recognitio...

0 Sheng Zhang, et al. ∙

research

∙ 08/11/2022

Structural Biases for Improving Transformers on Translation into Morphologically Rich Languages

Machine translation has seen rapid progress with the advent of Transform...

9 Paul Soulos, et al. ∙

research

∙ 08/11/2022

Interactive Code Generation via Test-Driven User-Intent Formalization

Pre-trained large language models (LLMs) such as OpenAI Codex have shown...

0 Shuvendu K. Lahiri, et al. ∙

research

∙ 06/24/2022

OPERA: Harmonizing Task-Oriented Dialogs and Information Seeking Experience

Existing studies in conversational AI mostly treat task-oriented dialog ...

0 Miaoran Li, et al. ∙

research

∙ 06/22/2022

GODEL: Large-Scale Pre-Training for Goal-Directed Dialog

We introduce GODEL (Grounded Open Dialogue Language Model), a large pre-...

0 Baolin Peng, et al. ∙

research

∙ 06/15/2022

Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone

Vision-language (VL) pre-training has recently received considerable att...

13 Zi-Yi Dou, et al. ∙

research

∙ 06/12/2022

GLIPv2: Unifying Localization and Vision-Language Understanding

We present GLIPv2, a grounded VL understanding model, that serves both l...

10 Haotian Zhang, et al. ∙

research

∙ 06/04/2022

Fault-Aware Neural Code Rankers

Large language models (LLMs) have demonstrated an impressive ability to ...

0 Jeevana Priya Inala, et al. ∙

research

∙ 05/28/2022

Learning from Self-Sampled Correct and Partially-Correct Programs

Program synthesis aims to generate executable programs that are consiste...

0 Ansong Ni, et al. ∙

research

∙ 05/24/2022

AdaMix: Mixture-of-Adapter for Parameter-efficient Tuning of Large Language Models

Fine-tuning large-scale pre-trained language models to downstream tasks ...

0 Yaqing Wang, et al. ∙

research

∙ 05/20/2022

Visually-Augmented Language Modeling

Human language is grounded on multimodal knowledge including visual know...

0 Weizhi Wang, et al. ∙

research

∙ 05/19/2022

Training Vision-Language Transformers from Captions Alone

We show that Vision-Language Transformers can be learned without human l...

0 Liangke Gui, et al. ∙

research

∙ 05/02/2022

Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systems

What explains the dramatic progress from 20th-century to 21st-century AI...

0 Paul Smolensky, et al. ∙

research

∙ 04/20/2022

K-LITE: Learning Transferable Visual Models with External Knowledge

Recent state-of-the-art computer vision systems are trained from natural...

3 Sheng Shen, et al. ∙

research

∙ 04/19/2022

ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models

Learning visual representations from natural language supervision has re...

2 Chunyuan Li, et al. ∙

research

∙ 04/16/2022

Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners

Traditional multi-task learning (MTL) methods use dense networks that us...

2 Shashank Gupta, et al. ∙

Jianfeng Gao

Featured Co-authors