A new approach to evaluating legibility: Comparing legibility frameworks using framework-independent robot motion trajectories

by   Sebastian Wallkotter, et al.

Robots that share an environment with humans may communicate their intent using a variety of different channels. Movement is one of these channels and, particularly in manipulation tasks, intent communication via movement is called legibility. It alters a robot's trajectory to make it intent expressive. Here we propose a novel evaluation method that improves the data efficiency of collected experimental data when benchmarking approaches generating such legible behavior. The primary novelty of the proposed method is that it uses trajectories that were generated independently of the framework being tested. This makes evaluation easier, enables N-way comparisons between approaches, and allows easier comparison across papers. We demonstrate the efficiency of the new evaluation method by comparing 10 legibility frameworks in 2 scenarios. The paper, thus, provides readers with (1) a novel approach to investigate and/or benchmark legibility, (2) an overview of existing frameworks, (3) an evaluation of 10 legibility frameworks (from 6 papers), and (4) evidence that viewing angle and trajectory progression matter when users evaluate the legibility of a motion.


page 2

page 3

page 10

page 12

page 15

page 19

page 20

page 21


How to Communicate Robot Motion Intent: A Scoping Review

Robots are becoming increasingly omnipresent in our daily lives, support...

Communicating Robot Arm Motion Intent Through Mixed Reality Head-mounted Displays

Efficient motion intent communication is necessary for safe and collabor...

Generating Human-Like Movement: A Comparison Between Two Approaches Based on Environmental Features

Modelling realistic human behaviours in simulation is an ongoing challen...

A System for Traded Control Teleoperation of Manipulation Tasks using Intent Prediction from Hand Gestures

This paper presents a teleoperation system that includes robot perceptio...

AMLB: an AutoML Benchmark

Comparing different AutoML frameworks is notoriously challenging and oft...

Comparison of Motion Encoding Frameworks on Human Manipulation Actions

Movement generation, and especially generalisation to unseen situations,...

Please sign up or login with your details

Forgot password? Click here to reset