Practice with uncertainty integration improves performance on a qualitatively different task and with new visualizations

Abstract

Background: Every day, people must reason with uncertain information to make decisions that affect their lives and affect the performance of their jobs and organizations. Visualizations of data uncertainty can facilitate these decisions, but visualizations are often misunderstood or misused. Previous research has demonstrated that deliberate practice with uncertainty visualizations can improve decision-making in abstract conditions, but it is not yet known whether the learning gains from this practice will transfer to more concrete, realistic, and complex decision-making tasks. Objective: Here, we test the degree to which practice integrating multiple sources of uncertain information with abstract 2-d summary or ensemble displays improves performance on a similar transfer task involving decision-making with a 3-d virtual sand table. Method: We conducted an online study with 378 participants who completed an uncertainty integration task in a 3-d virtual sand table context using either summary or ensemble displays of uncertainty. Participants had previously practiced with the same display, the other display, or received no opportunity to practice. We analyzed response accuracy and speed and how they changed throughout the task. Results: Results suggest that deliberate practice with abstract uncertainty visualizations allows faster decision making in the new context but does not improve accuracy. In the 3-d task, the summary display generally yielded similar or better performance than the ensemble display. Learning gains from practice transferred to both same-type and different-type visualizations in the 3-d condition. Conclusions: The results suggest that practice in the 2-d task enhanced facility with the underlying probabilistic reasoning in a new context rather than just increasing visualization-specific understanding. This implies that deliberate practice can be a beneficial tool to improve reasoning with uncertainty, including across contexts and across visualization types. Materials: Stimuli, stimulus software, anonymized data, and analysis scripts and related code are available online at https://osf.io/5xdsg/?view_only=8d422629a3784f6a80cfeae40e59a078