research
∙
06/28/2023
MultiZoo MultiBench: A Standardized Toolkit for Multimodal Deep Learning
Learning multimodal representations involves integrating information fro...
share
research
∙
06/13/2023
Neural Mixed Effects for Nonlinear Personalized Predictions
Personalized prediction is a machine learning approach that predicts a p...
research
∙
06/08/2023
Factorized Contrastive Learning: Going Beyond Multi-view Redundancy
In a wide range of multimodal tasks, contrastive learning has become a p...
research
∙
06/07/2023
Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions
Societal biases present in pre-trained large language models are a criti...
research
∙
06/07/2023
Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applications
In many machine learning systems that jointly learn from multiple modali...
research
∙
06/07/2023
Multimodal Fusion Interactions: A Study of Human and Automatic Quantification
Multimodal fusion of multiple heterogeneous and interconnected signals i...
research
∙
05/24/2023
SenteCon: Leveraging Lexicons to Learn Human-Interpretable Language Representations
Although deep language representations have become the dominant form of ...
research
∙
05/23/2023
Difference-Masking: Choosing What to Mask in Continued Pretraining
Self-supervised learning (SSL) and the objective of masking-and-predicti...
research
∙
05/23/2023
Counterfactual Augmentation for Multimodal Learning Under Presentation Bias
In real-world machine learning systems, labels are often derived from us...
research
∙
05/18/2023
Expanding the Role of Affective Phenomena in Multimodal Interaction Research
In recent decades, the field of affective computing has made substantial...
research
∙
02/23/2023
Quantifying Modeling Feature Interactions: An Information Decomposition Framework
The recent explosion of interest in multimodal applications has resulted...
research
∙
12/20/2022
Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment
Despite recent progress towards scaling up multimodal vision-language mo...
research
∙
11/23/2022
SeedBERT: Recovering Annotator Rating Distributions from an Aggregated Label
Many machine learning tasks – particularly those in affective computing ...
research
∙
11/10/2022
Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control
Pretrained language models have demonstrated extraordinary capabilities ...
research
∙
09/25/2022
Paraphrasing Is All You Need for Novel Object Captioning
Novel object captioning (NOC) aims to describe images containing objects...
research
∙
09/07/2022
Foundations and Recent Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions
Multimodal machine learning is a vibrant multi-disciplinary research fie...
research
∙
08/17/2022
Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides
Lecture slide presentations, a sequence of pages that contain text and f...
research
∙
07/29/2022
Face-to-Face Contrastive Learning for Social Intelligence Question-Answering
Creating artificial social intelligence - algorithms that can understand...
research
∙
06/30/2022
MultiViz: An Analysis Benchmark for Visualizing and Understanding Multimodal Models
The promise of multimodal models for real-world applications has inspire...
research
∙
03/21/2022
PACS: A Dataset for Physical Audiovisual CommonSense Reasoning
In order for AI to be safely deployed in real-world scenarios such as ho...
research
∙
03/03/2022
DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations
The ability for a human to understand an Artificial Intelligence (AI) mo...
research
∙
03/02/2022
HighMMT: Towards Modality and Task Generalization for High-Modality Representation Learning
Learning multimodal representations involves discovering correspondences...
research
∙
02/14/2022
Learning Weakly-Supervised Contrastive Representations
We argue that a form of the valuable information provided by the auxilia...
research
∙
02/11/2022
Conditional Contrastive Learning with Kernel
Conditional contrastive learning frameworks consider the conditional sam...
research
∙
10/26/2021
Relay Variational Inference: A Method for Accelerated Encoderless VI
Variational Inference (VI) offers a method for approximating intractable...
research
∙
07/28/2021
Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis
Multimodal sentiment analysis aims to extract and integrate semantic inf...
research
∙
07/15/2021
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
Learning multimodal representations involves integrating information fro...
research
∙
06/24/2021
Towards Understanding and Mitigating Social Biases in Language Models
As machine learning methods are deployed in real-world settings such as ...
research
∙
06/24/2021
Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data
Mental health conditions remain underdiagnosed even in countries with co...
research
∙
06/05/2021
Integrating Auxiliary Information in Self-supervised Learning
This paper presents to integrate the auxiliary information (e.g., additi...
research
∙
06/05/2021
Conditional Contrastive Learning: Removing Undesirable Information in Self-Supervised Representations
Self-supervised learning is a form of unsupervised learning that leverag...
research
∙
04/28/2021
A Note on Connecting Barlow Twins with Negative-Sample-Free Contrastive Learning
In this report, we relate the algorithmic design of Barlow Twins' method...
research
∙
03/21/2021
Self-supervised Representation Learning with Relative Predictive Coding
This paper introduces Relative Predictive Coding (RPC), a new contrastiv...
research
∙
01/22/2021
Understanding the Tradeoffs in Client-Side Privacy for Speech Recognition
Existing approaches to ensuring privacy of user speech data primarily fo...
research
∙
01/03/2021
StarNet: Gradient-free Training of Deep Generative Models using Determined System of Linear Equations
In this paper we present an approach for training deep generative models...
research
∙
12/04/2020
Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment
The natural world is abundant with concepts expressed via visual, acoust...
research
∙
12/04/2020
Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary Study
Mental health conditions remain under-diagnosed even in countries with c...
research
∙
10/27/2020
Unsupervised Domain Adaptation for Visual Navigation
Advances in visual navigation methods have led to intelligent embodied n...
research
∙
10/22/2020
MTGAT: Multimodal Temporal Graph Attention Networks for Unaligned Human Multimodal Language Sequences
Human communication is multimodal in nature; it is through multiple moda...
research
∙
08/31/2020
Toward Multimodal Modeling of Emotional Expressiveness
Emotional expressiveness captures the extent to which a person tends to ...
research
∙
07/24/2020
Style Transfer for Co-Speech Gesture Animation: A Multi-Speaker Conditional-Mixture Approach
How can we teach robots or virtual assistants to gesture naturally? Can ...
research
∙
07/16/2020
Towards Debiasing Sentence Representations
As natural language processing methods are increasingly deployed in real...
research
∙
07/07/2020
What Gives the Answer Away? Question Answering Bias Analysis on Video QA Datasets
Question answering biases in video QA datasets can mislead multimodal mo...
research
∙
06/10/2020
Demystifying Self-Supervised Learning: An Information-Theoretical Framework
Self-supervised representation learning adopts self-defined signals as s...
research
∙
06/09/2020
Neural Methods for Point-wise Dependency Estimation
Since its inception, the neural estimation of mutual information (MI) ha...
research
∙
05/03/2020
Improving Aspect-Level Sentiment Analysis with Aspect Extraction
Aspect-based sentiment analysis (ABSA), a popular research area in NLP h...
research
∙
04/29/2020
Interpretable Multimodal Routing for Human Multimodal Language
The human language has heterogeneous sources of information, including t...
research
∙
03/06/2020
Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding
Multi-agent trajectory forecasting in autonomous driving requires an age...
research
∙
03/04/2020
On Emergent Communication in Competitive Multi-Agent Teams
Several recent works have found the emergence of grounded compositional ...
research
∙
02/16/2020