Can you hear me now? Sensitive comparisons of human and machine perception

by   Michael A Lepori, et al.

The rise of sophisticated machine-recognition systems has brought with it a rise in comparisons between human and machine perception. But such comparisons face an asymmetry: Whereas machine perception of some stimulus can often be probed through direct and explicit measures, much of human perceptual knowledge is latent, incomplete, or embedded in unconscious mental processes that may not be available for explicit report. Here, we show how this asymmetry can cause such comparisons to underestimate the overlap in human and machine perception. As a case study, we consider human perception of adversarial speech– synthetic audio commands that are recognized as valid messages by automated speech-recognition systems but that human listeners reportedly hear as meaningless noise. In five experiments, we adapt task designs from the human psychophysics literature to show that even when subjects cannot freely transcribe adversarial speech (the previous benchmark for human understanding), they nevertheless can discriminate adversarial speech from closely matched non-speech (Experiments 1-2), finish common phrases begun in adversarial speech (Experiments 3-4), and solve simple math problems posed in adversarial speech (Experiment 5) – even for stimuli previously described as "unintelligible to human listeners". We recommend the adoption of sensitive tests of human and machine perception, and discuss the broader consequences of this approach for comparing natural and artificial intelligence.


page 1

page 2

page 3

page 4


Audio Adversarial Examples: Attacks Using Vocal Masks

We construct audio adversarial examples on automatic Speech-To-Text syst...

BWSNet: Automatic Perceptual Assessment of Audio Signals

This paper introduces BWSNet, a model that can be trained from raw human...

Perception-Aware Attack: Creating Adversarial Music via Reverse-Engineering Human Perception

Recently, adversarial machine learning attacks have posed serious securi...

Guiding Machine Perception with Psychophysics

Gustav Fechner's 1860 delineation of psychophysics, the measurement of s...

Accented Speech Recognition Inspired by Human Perception

While improvements have been made in automatic speech recognition perfor...

Perception Lie Paradox: Mathematically Proved Uncertainty about Humans Perception Similarity

Agents' judgment depends on perception and previous knowledge. Assuming ...

A Surprising Density of Illusionable Natural Speech

Recent work on adversarial examples has demonstrated that most natural i...

Please sign up or login with your details

Forgot password? Click here to reset