Metrics to select design tasks in experimental creativity research

Abstract

This paper demonstrates the need for systematic approaches to assess design tasks for experimental studies of idea generation and conceptual design. We first examine a collection of 160 design tasks used in the literature revealing a high variance of task characteristics. Variance across studies is captured by analysing operating variables including task elaboration, task orientation, task selection, participants and time allocated for ideation. Four metrics are then proposed to assist researchers to objectively characterise design tasks. These metrics help identify the semantic and lexical dimensions of problem statements, evaluate precedent ideas and assess task wording. The value of this approach is demonstrated by applying the metrics to a subset of design tasks showing their usefulness to assess the design task objectively as part of the wider experimental design process. Armed with these metrics, researchers can tune both the task and the wider experiment to best match the purposes and the operating variables of their studies. The paper concludes by formulating concrete recommendations to assist in the selection of design tasks using these metrics.