How Many Data Samples is an Additional Instruction Worth?

03/17/2022
by   Ravsehaj Singh Puri, et al.
4

Recently introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. Instruction-tuned models have significantly outperformed multitask learning models (without instruction); however they are far from state of the art task specific models. Conventional approaches to improve model performance via creating large datasets with lots of task instances or architectural/training changes in model may not be feasible for non-expert users. However, they can write alternate instructions to represent an instruction task. Is Instruction-augumentation helpful? We augment a subset of tasks in the expanded version of NATURAL INSTRUCTIONS with additional instructions and find that these significantly improve model performance (up to 35 low-data regime. Our results indicate that an additional instruction can be equivalent to  200 data samples on average across tasks.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset