b'Paul Pu Liang'

research

∙ 08/27/2023

Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP

Mechanistic interpretability seeks to understand the neural mechanisms t...

0 Vedant Palit, et al. ∙

research

∙ 06/28/2023

MultiZoo MultiBench: A Standardized Toolkit for Multimodal Deep Learning

Learning multimodal representations involves integrating information fro...

0 Paul Pu Liang, et al. ∙

research

∙ 06/08/2023

Factorized Contrastive Learning: Going Beyond Multi-view Redundancy

In a wide range of multimodal tasks, contrastive learning has become a p...

0 Paul Pu Liang, et al. ∙

research

∙ 06/07/2023

Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions

Societal biases present in pre-trained large language models are a criti...

0 Himanshu Thakur, et al. ∙

research

∙ 06/07/2023

Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applications

In many machine learning systems that jointly learn from multiple modali...

6 Paul Pu Liang, et al. ∙

research

∙ 06/07/2023

Multimodal Fusion Interactions: A Study of Human and Automatic Quantification

Multimodal fusion of multiple heterogeneous and interconnected signals i...

0 Paul Pu Liang, et al. ∙

research

∙ 05/23/2023

Difference-Masking: Choosing What to Mask in Continued Pretraining

Self-supervised learning (SSL) and the objective of masking-and-predicti...

0 Alex Wilf, et al. ∙

research

∙ 05/21/2023

HIINT: Historical, Intra- and Inter- personal Dynamics Modeling with Cross-person Memory Transformer

Accurately modeling affect dynamics, which refers to the changes and flu...

0 Yubin Kim, et al. ∙

research

∙ 02/23/2023

Quantifying Modeling Feature Interactions: An Information Decomposition Framework

The recent explosion of interest in multimodal applications has resulted...

0 Paul Pu Liang, et al. ∙

research

∙ 02/09/2023

Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals

High sample complexity has long been a challenge for RL. On the other ha...

0 Yue Wu, et al. ∙

research

∙ 12/20/2022

Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment

Despite recent progress towards scaling up multimodal vision-language mo...

9 rohan-pandey, et al. ∙

research

∙ 11/10/2022

Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control

Pretrained language models have demonstrated extraordinary capabilities ...

0 Xiang Fan, et al. ∙

research

∙ 09/07/2022

Foundations and Recent Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions

Multimodal machine learning is a vibrant multi-disciplinary research fie...

7 Paul Pu Liang, et al. ∙

research

∙ 08/17/2022

Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides

Lecture slide presentations, a sequence of pages that contain text and f...

26 Dong Won Lee, et al. ∙

research

∙ 07/29/2022

Face-to-Face Contrastive Learning for Social Intelligence Question-Answering

Creating artificial social intelligence - algorithms that can understand...

7 Alex Wilf, et al. ∙

research

∙ 06/30/2022

MultiViz: An Analysis Benchmark for Visualizing and Understanding Multimodal Models

The promise of multimodal models for real-world applications has inspire...

2 Paul Pu Liang, et al. ∙

research

∙ 03/21/2022

PACS: A Dataset for Physical Audiovisual CommonSense Reasoning

In order for AI to be safely deployed in real-world scenarios such as ho...

4 Samuel Yu, et al. ∙

research

∙ 03/03/2022

DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations

The ability for a human to understand an Artificial Intelligence (AI) mo...

8 Yiwei Lyu, et al. ∙

research

∙ 03/02/2022

HighMMT: Towards Modality and Task Generalization for High-Modality Representation Learning

Learning multimodal representations involves discovering correspondences...

10 Paul Pu Liang, et al. ∙

research

∙ 07/15/2021

MultiBench: Multiscale Benchmarks for Multimodal Representation Learning

Learning multimodal representations involves integrating information fro...

0 Paul Pu Liang, et al. ∙

research

∙ 06/24/2021

Towards Understanding and Mitigating Social Biases in Language Models

As machine learning methods are deployed in real-world settings such as ...

16 Paul Pu Liang, et al. ∙

research

∙ 06/24/2021

Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data

Mental health conditions remain underdiagnosed even in countries with co...

12 Paul Pu Liang, et al. ∙

research

∙ 04/24/2021

Ask Explore: Grounded Question Answering for Curiosity-Driven Exploration

In many real-world scenarios where extrinsic rewards to the agent are ex...

0 Jivat Neet Kaur, et al. ∙

research

∙ 01/22/2021

Understanding the Tradeoffs in Client-Side Privacy for Speech Recognition

Existing approaches to ensuring privacy of user speech data primarily fo...

0 Peter Wu, et al. ∙

research

∙ 12/04/2020

Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment

The natural world is abundant with concepts expressed via visual, acoust...

6 Paul Pu Liang, et al. ∙

research

∙ 12/04/2020

Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary Study

Mental health conditions remain under-diagnosed even in countries with c...

3 Terrance Liu, et al. ∙

research

∙ 10/23/2020

An Investigation of how Label Smoothing Affects Generalization

It has been hypothesized that label smoothing can reduce overfitting and...

0 Blair Chen, et al. ∙

research

∙ 07/16/2020

Towards Debiasing Sentence Representations

As natural language processing methods are increasingly deployed in real...

0 Paul Pu Liang, et al. ∙

research

∙ 03/18/2020

Anchor Transform: Learning Sparse Representations of Discrete Objects

Learning continuous representations of discrete objects such as text, us...

0 Paul Pu Liang, et al. ∙

research

∙ 03/06/2020

Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding

Multi-agent trajectory forecasting in autonomous driving requires an age...

9 Seong Hyeon Park, et al. ∙

research

∙ 03/04/2020

On Emergent Communication in Competitive Multi-Agent Teams

Several recent works have found the emergence of grounded compositional ...

10 Paul Pu Liang, et al. ∙

research

∙ 02/16/2020

Learning Not to Learn in the Presence of Noisy Labels

Learning in the presence of label noise is a challenging yet important t...

11 Liu Ziyin, et al. ∙

research

∙ 01/06/2020

Think Locally, Act Globally: Federated Learning with Local and Global Representations

Federated learning is an emerging research paradigm to train models on p...

6 Paul Pu Liang, et al. ∙

research

∙ 11/22/2019

Factorized Multimodal Transformer for Multimodal Sequential Learning

The complex world around us is inherently multimodal and sequential (con...

31 Amir Zadeh, et al. ∙

research

∙ 07/01/2019

Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization

There has been an increased interest in multimodal language processing i...

17 Paul Pu Liang, et al. ∙

research

∙ 06/29/2019

Deep Gamblers: Learning to Abstain with Portfolio Theory

We deal with the selective classification problem (supervised-learning p...

8 Liu Ziyin, et al. ∙

research

∙ 06/01/2019

Multimodal Transformer for Unaligned Multimodal Language Sequences

Human language is often multimodal, which comprehends a mixture of natur...

0 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 05/14/2019

Strong and Simple Baselines for Multimodal Utterance Embeddings

Human language is a rich multimodal signal consisting of spoken words, f...

0 Paul Pu Liang, et al. ∙

research

∙ 03/03/2019

Variational Auto-Decoder: Neural Generative Modeling from Partial Data

Learning a generative model from partial data (data with missingness) is...

32 Amir Zadeh, et al. ∙

research

∙ 03/03/2019

Variational Auto-Decoder

Learning a generative model from partial data (data with missingness) is...

0 Amir Zadeh, et al. ∙

research

∙ 12/19/2018

An Empirical Evaluation of Sketched SVD and its Application to Leverage Score Ordering

The power of randomized algorithms in numerical methods have led to fast...

0 Hui Han Chin, et al. ∙

research

∙ 12/19/2018

Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities

Multimodal sentiment analysis is a core research area that studies speak...

0 Hai Pham, et al. ∙

research

∙ 11/23/2018

Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors

Humans convey their intentions through the usage of both verbal and nonv...

0 Yansen Wang, et al. ∙

research

∙ 08/12/2018

Multimodal Language Analysis with Recurrent Multistage Fusion

Computational modeling of human multimodal language is an emerging resea...

0 Paul Pu Liang, et al. ∙

research

∙ 08/12/2018

Multimodal Local-Global Ranking Fusion for Emotion Recognition

Emotion recognition is a core research area at the intersection of artif...

0 Paul Pu Liang, et al. ∙

research

∙ 07/11/2018

Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis

Multimodal machine learning is a core research area spanning the languag...

0 Hai Pham, et al. ∙

research

∙ 06/16/2018

Learning Factorized Multimodal Representations

Learning representations of multimodal data is a fundamentally complex r...

0 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 05/31/2018

Efficient Low-rank Multimodal Fusion with Modality-Specific Factors

Multimodal research is an emerging field of artificial intelligence, and...

0 Zhun Liu, et al. ∙

research

∙ 02/03/2018

Memory Fusion Network for Multi-view Sequential Learning

Multi-view sequential learning is a fundamental problem in machine learn...

0 Amir Zadeh, et al. ∙

research

∙ 02/03/2018

Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning

With the increasing popularity of video sharing websites such as YouTube...

0 Minghai Chen, et al. ∙

Paul Pu Liang

Featured Co-authors