Joint Activity Design Evaluation Tools for Conducting Staged-World Studies

Abstract

Intelligence analysis requires an analyst to search for meaningful events in vast amounts of distributed data and synthesize those events into insights to inform decision-making. Because of the complexity, dynamics of the ecology, and typical functions of work, analysis performance can be difficult to study in traditional laboratory environments. Staged-world studies, which maintain experimental control while preserving the complexity of the work domain, represent a promising approach to study analyst performance. To support an ongoing research program, we are developing the Joint Activity Design Evaluation (JADE) platform, a software tool for conducting staged-world studies within the intelligence analysis (IA) domain. Herein, we describe three of JADE’s core capabilities for presenting scenarios based on common challenges of cognitive work, eliciting participant responses, and storing and displaying pertinent data. As JADE continues to mature and add new features, we expect it to become a tool for researchers to conduct staged-world studies across a broader range of application domains.