Active Learning for Event Extraction with Memory-based Loss Prediction Model

by   Shirong Shen, et al.
Southeast University

Event extraction (EE) plays an important role in many industrial application scenarios, and high-quality EE methods require a large amount of manual annotation data to train supervised learning models. However, the cost of obtaining annotation data is very high, especially for annotation of domain events, which requires the participation of experts from corresponding domain. So we introduce active learning (AL) technology to reduce the cost of event annotation. But the existing AL methods have two main problems, which make them not well used for event extraction. Firstly, the existing pool-based selection strategies have limitations in terms of computational cost and sample validity. Secondly, the existing evaluation of sample importance lacks the use of local sample information. In this paper, we present a novel deep AL method for EE. We propose a batch-based selection strategy and a Memory-Based Loss Prediction model (MBLP) to select unlabeled samples efficiently. During the selection process, we use an internal-external sample loss ranking method to evaluate the sample importance by using local information. Finally, we propose a delayed training strategy to train the MBLP model. Extensive experiments are performed on three domain datasets, and our method outperforms other state-of-the-art methods.


Temporal Output Discrepancy for Loss Estimation-based Active Learning

While deep learning succeeds in a wide range of tasks, it highly depends...

Iterative Loop Learning Combining Self-Training and Active Learning for Domain Adaptive Semantic Segmentation

Recently, self-training and active learning have been proposed to allevi...

Confidence-Aware Active Feedback for Efficient Instance Search

Relevance feedback is widely used in instance search (INS) tasks to furt...

Boosting Backdoor Attack with A Learnable Poisoning Sample Selection Strategy

Data-poisoning based backdoor attacks aim to insert backdoor into models...

A Survey of Deep Active Learning

Active learning (AL) attempts to maximize the performance gain of the mo...

Tyger: Task-Type-Generic Active Learning for Molecular Property Prediction

How to accurately predict the properties of molecules is an essential pr...

A Practical Incremental Learning Framework For Sparse Entity Extraction

This work addresses challenges arising from extracting entities from tex...

Please sign up or login with your details

Forgot password? Click here to reset