The Human Evaluation Datasheet 1.0: A Template for Recording Details of Human Evaluation Experiments in NLP

03/17/2021
by   Anastasia Shimorina, et al.
13

This paper introduces the Human Evaluation Datasheet, a template for recording the details of individual human evaluation experiments in Natural Language Processing (NLP). Originally taking inspiration from seminal papers by Bender and Friedman (2018), Mitchell et al. (2019), and Gebru et al. (2020), the Human Evaluation Datasheet is intended to facilitate the recording of properties of human evaluations in sufficient detail, and with sufficient standardisation, to support comparability, meta-evaluation, and reproducibility tests.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset