Development of a usability checklist for public health dashboards to identify violations of usability principles

Abstract

Abstract Objective To develop a usability checklist for public health dashboards. Materials and methods This study systematically evaluated all publicly available dashboards for sexually transmitted infections on state health department websites in the United States (N = 13). A set of 11 principles derived from the information visualization literature were used to identify usability problems that violate critical usability principles: spatial organization, information coding, consistency, removal of extraneous ink, recognition rather than recall, minimal action, dataset reduction, flexibility to user experience, understandability of contents, scientific integrity, and readability. Three user groups were considered for public health dashboards: public health practitioners, academic researchers, and the general public. Six reviewers with usability knowledge and diverse domain expertise examined the dashboards using a rubric based on the 11 principles. Data analysis included quantitative analysis of experts’ usability scores and qualitative synthesis of their textual comments. Results The dashboards had varying levels of complexity, and the usability scores were dependent on the dashboards’ complexity. Overall, understandability of contents, flexibility, and scientific integrity were the areas with the most major usability problems. The usability problems informed a checklist to improve performance in the 11 areas. Discussion The varying complexity of the dashboards suggests a diversity of target audiences. However, the identified usability problems suggest that dashboards’ effectiveness for different groups of users was limited. Conclusions The usability of public health data dashboards can be improved to accommodate different user groups. This checklist can guide the development of future public health dashboards to engage diverse audiences.