Vision Meets Definitions: Unsupervised Visual Word Sense Disambiguation Incorporating Gloss Information

by   Sunjae Kwon, et al.
UMass Lowell
University of Massachusetts Amherst

Visual Word Sense Disambiguation (VWSD) is a task to find the image that most accurately depicts the correct sense of the target word for the given context. Previously, image-text matching models often suffered from recognizing polysemous words. This paper introduces an unsupervised VWSD approach that uses gloss information of an external lexical knowledge-base, especially the sense definitions. Specifically, we suggest employing Bayesian inference to incorporate the sense definitions when sense information of the answer is not provided. In addition, to ameliorate the out-of-dictionary (OOD) issue, we propose a context-aware definition generation with GPT-3. Experimental results show that the VWSD performance significantly increased with our Bayesian inference-based approach. In addition, our context-aware definition generation achieved prominent performance improvement in OOD examples exhibiting better performance than the existing definition generation method. We will publish source codes as soon as possible.


Multi-sense Definition Modeling using Word Sense Decompositions

Word embeddings capture syntactic and semantic information about words. ...

Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis

We propose using automatically generated natural language definitions of...

Context-Aware Semantic Similarity Measurement for Unsupervised Word Sense Disambiguation

The issue of word sense ambiguity poses a significant challenge in natur...

A Spatiotemporal Context Definition for Service Adaptation Prediction in a Pervasive Computing Environment

Pervasive systems refers to context-aware systems that can sense their c...

Term Definitions Help Hypernymy Detection

Existing methods of hypernymy detection mainly rely on statistics over a...

Measuring diachronic sense change: new models and Monte Carlo methods for Bayesian inference

In a bag-of-words model, the senses of a word with multiple meanings, e....

Evaluating a Multi-sense Definition Generation Model for Multiple Languages

Most prior work on definition modeling has not accounted for polysemy, o...

Please sign up or login with your details

Forgot password? Click here to reset