On Human Visual Contrast Sensitivity and Machine Vision Robustness: A Comparative Study

by   Ming-Chang Chiu, et al.

It is well established in neuroscience that color vision plays an essential part in the human visual perception system. Meanwhile, many novel designs for computer vision inspired by human vision have achieved success in a wide range of tasks and applications. Nonetheless, how color differences affect machine vision has not been well explored. Our work tries to bridge this gap between the human color vision aspect of visual recognition and that of the machine. To achieve this, we curate two datasets: CIFAR10-F and CIFAR100-F, which are based on the foreground colors of the popular CIFAR datasets. Together with CIFAR10-B and CIFAR100-B, the existing counterpart datasets with information on the background colors of CIFAR test sets, we assign each image based on its color contrast level per its foreground and background color labels and use this as a proxy to study how color contrast affects machine vision. We first conduct a proof-of-concept study, showing the effect of color difference and validate our datasets. Furthermore, on a broader level, an important characteristic of human vision is its robustness against ambient changes; therefore, drawing inspirations from ophthalmology and the robustness literature, we analogize contrast sensitivity from the human visual aspect to machine vision and complement the current robustness study using corrupted images with our CIFAR-CoCo datasets. In summary, motivated by neuroscience and equipped with the datasets we curate, we devise a new framework in two dimensions to perform extensive analyses on the effect of color contrast and corrupted images: (1) model architecture, (2) model size, to measure the perception ability of machine vision beyond total accuracy. We also explore how task complexity and data augmentation play a role in this setup. Our results call attention to new evaluation approaches for human-like machine perception.


page 2

page 3

page 4

page 5

page 6

page 7

page 9

page 12


A Computational Model of Afterimages based on Simultaneous and Successive Contrasts

Negative afterimage appears in our vision when we shift our gaze from an...

Divergences in Color Perception between Deep Neural Networks and Humans

Deep neural networks (DNNs) are increasingly proposed as models of human...

Edge-Aware Image Color Appearance and Difference Modeling

The perception of color is one of the most important aspects of human vi...

Visual Illusions Also Deceive Convolutional Neural Networks: Analysis and Implications

Visual illusions allow researchers to devise and test new models of visu...

BOLD5000: A public fMRI dataset of 5000 images

Vision science, particularly machine vision, has been revolutionized by ...

It is hard to see a needle in a haystack: Modeling contrast masking effect in a numerical observer

Within the framework of a virtual clinical trial for breast imaging, we ...

Please sign up or login with your details

Forgot password? Click here to reset