Construct Development and Validation in Game-Based Research

Abstract

Problem. Teams or groups of individuals working together to achieve a shared goal, make up today’s world of work. Although the literature is rife with issues concerning teams, there is no coherent structure to guide researchers wishing to gain a deeper understanding into those factors leading to positive team outcomes. Question. This is due in part to two factors; one being the methods (e.g., observation; self report) typically employed to study teams often do not apply rigorous standards for reliability and validity. The second is it is difficult to construct data gathering situations that realistically approach the context in which teams operate. Approach. Addressing the first issue, we present a framework for the type of data that should be gathered to reliably and validly evaluate team performance. We believe the second issue is best addressed through the application of serious games, where realistic scenarios are delivered representing problems similar to those faced by teams in the real world. Finally, we describe a study whereby we demonstrate the approach utilizing a serious game to gather the data, which is then analyzed to assess the reliability and validity of the measures. Conclusion. With reliability and validity being established, a latent change score model is presented to illustrate how rich models of team interaction can be stated, investigated, and statistically assessed; since the serious game ensures the quality of the team interaction and the superior quality of the data.