Toward automatic comparison of visualization techniques: Application to graph visualization

by   R. Bourqui, et al.

Many end-user evaluations of data visualization techniques have been run during the last decades. Their results are cornerstones to build efficient visualization systems. However, designing an evaluation is always complex and time-consuming and may end in a lack of statistical evidence. The raising of modern efficient computer vision techniques may help visualization researchers to adjust their evaluation hypothesis and thus reduces the risk of failure. In this paper, we present a methodology that uses such computer vision techniques to automatically compare the efficiency of several visualization techniques. The basis of our methodology is to generate a set of images for each compared visualization technique from a common dataset and to train machine learning models (one for each set and visualization technique) to solve a given task. Our assumption is that the performance of each model allows to compare the efficiencies of the corresponding visualization techniques; as current machine learning models are not capable enough to reflect human capabilities, including their imperfections, such results should be interpreted with caution. However, we argue that using machine learning-based evaluation as a pre-process of standard user evaluations should help researchers to perform a more exhaustive study of the design space and thus should improve the final user evaluation by providing better test cases. To show that our methodology can reproduce, up to a certain level, results of user evaluations, we applied it to compare two mainstream graph visualization techniques: node-link (NL) and adjacency-matrix (MD) diagrams. We partially reproduced a user evaluation from Ghoniem et al. using two well-known deep convolutional neural networks as machine learning-based systems. Our results show up that Ghoniem et al. results can be reproduced automatically at a larger scale with our system.


page 7

page 11


Inferential Tasks as an Evaluation Technique for Visualization

Designing suitable tasks for visualization evaluation remains challengin...

DeepHashing using TripletLoss

Hashing is one of the most efficient techniques for approximate nearest ...

A Tour of Visualization Techniques for Computer Vision Datasets

We survey a number of data visualization techniques for analyzing Comput...

What Do We Actually Learn from Evaluations in the "Heroic Era" of Visualization?

We often point to the relative increase in the amount and sophistication...

Perceptual Pat: A Virtual Human System for Iterative Visualization Design

Designing a visualization is often a process of iterative refinement whe...

Toward Multimodal Interaction in Scalable Visual Digital Evidence Visualization Using Computer Vision Techniques and ISS

Visualization requirements in Forensic Lucid have to do with different l...

Algorithmic failure as a humanities methodology: machine learning's mispredictions identify rich cases for qualitative analysis

This commentary tests a methodology proposed by Munk et al. (2022) for u...

Please sign up or login with your details

Forgot password? Click here to reset