A comparison of benchmark task and insight evaluation methods for information visualization

Abstract

This study compares two different empirical research methods for evaluating information visualizations: the traditional benchmark-task method and the insight method. The methods are compared using criteria such as the conclusions about the visualization designs provided by each method, the time participants spent during the study, the time and effort required to analyse the resulting empirical data, and the effect of individual differences between participants on the results. The study compares three graph visualization alternatives that associate bioinformatics microarray time series data to pathway graph vertices in order to investigate the effect of different visual grouping structures in visualization designs that integrate multiple data types. It is confirmed that visual grouping should match task structure, but interactive grouping proves to be a well-rounded alternative. Overall, the results validate the insight method’s ability to confirm results of the task method, but also show advantages of the insight method to illuminate additional types of tasks. Efficiency and insight frequently correlate, but important distinctions are found. Categories: H.5.2 [Information Interfaces and Presentation]: User Interfaces – evaluation/methodology.