Towards Better Entity Linking with Multi-View Enhanced Distillation

by   YI LIU, et al.

Dense retrieval is widely used for entity linking to retrieve entities from large-scale knowledge bases. Mainstream techniques are based on a dual-encoder framework, which encodes mentions and entities independently and calculates their relevances via rough interaction metrics, resulting in difficulty in explicitly modeling multiple mention-relevant parts within entities to match divergent mentions. Aiming at learning entity representations that can match divergent mentions, this paper proposes a Multi-View Enhanced Distillation (MVD) framework, which can effectively transfer knowledge of multiple fine-grained and mention-relevant parts within entities from cross-encoders to dual-encoders. Each entity is split into multiple views to avoid irrelevant information being over-squashed into the mention-relevant view. We further design cross-alignment and self-alignment mechanisms for this framework to facilitate fine-grained knowledge distillation from the teacher model to the student model. Meanwhile, we reserve a global-view that embeds the entity as a whole to prevent dispersal of uniform information. Experiments show our method achieves state-of-the-art performance on several entity linking benchmarks.


MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations

Entity retrieval, which aims at disambiguating mentions to canonical ent...

Entity Linking via Dual and Cross-Attention Encoders

Entity Linking has two main open areas of research: 1) generate candidat...

Learning Cross-Context Entity Representations from Text

Language modeling tasks, in which words, or word-pieces, are predicted o...

Fine-Grained Distillation for Long Document Retrieval

Long document retrieval aims to fetch query-relevant documents from a la...

ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrieval

Neural retrievers based on pre-trained language models (PLMs), such as d...

Multi-Grained Multimodal Interaction Network for Entity Linking

Multimodal entity linking (MEL) task, which aims at resolving ambiguous ...

Knowledge Enhanced Contextual Word Representations

Contextual word representations, typically trained on unstructured, unla...

Please sign up or login with your details

Forgot password? Click here to reset