EASE: An Easily-Customized Annotation System Powered by Efficiency Enhancement Mechanisms

by   Naihao Deng, et al.

The performance of current supervised AI systems is tightly connected to the availability of annotated datasets. Annotations are usually collected through annotation tools, which are often designed for specific tasks and are difficult to customize. Moreover, existing annotation tools with an active learning mechanism often only support limited use cases. To address these limitations, we present EASE, an Easily-Customized Annotation System Powered by Efficiency Enhancement Mechanisms. provides modular annotation units for building customized annotation interfaces and also provides multiple back-end options that suggest annotations using (1) multi-task active learning; (2) demographic feature based active learning; (3) a prompt system that can query the API of large language models. We conduct multiple experiments and user studies to evaluate our system's flexibility and effectiveness. Our results show that our system can meet the diverse needs of NLP researchers and significantly accelerate the annotation process.


page 11

page 12

page 14

page 16

page 17

page 19

page 20


ALANNO: An Active Learning Annotation System for Mortals

In today's data-driven society, supervised machine learning is rapidly e...

Multi-task Active Learning for Pre-trained Transformer-based Models

Multi-task learning, in which several tasks are jointly learned by a sin...

The DALPHI annotation framework & how its pre-annotations can improve annotator efficiency

Producing the required amounts of training data for machine learning and...

HUMAN: Hierarchical Universal Modular ANnotator

A lot of real-world phenomena are complex and cannot be captured by sing...

Active Learning for Coreference Resolution using Discrete Annotation

We improve upon pairwise annotation for active learning in coreference r...

Overview of Annotation Creation: Processes & Tools

Creating linguistic annotations requires more than just a reliable annot...

ActiveGLAE: A Benchmark for Deep Active Learning with Transformers

Deep active learning (DAL) seeks to reduce annotation costs by enabling ...

Please sign up or login with your details

Forgot password? Click here to reset