Confidence in Models and Simulations: A Multi-Stakeholder Analysis

Abstract

System stakeholders from multiple disciplines increasingly interact with computer models and simulations to make critical decisions. Advanced digital models have transformed how engineers interact, analyze requirements, develop and verify system elements, and test them to validate that they meet stakeholder needs. However, these models are often developed by a specific discipline for its own purposes. Convincing other stakeholders to accept the results of these tools can be a challenge, and indeed, the adoption of models and simulations at the level of system development still lags the pace of the underlying computational and application advances. The acceptance of models and simulations remains largely a function of the subjective preferences of engineers and other stakeholders. In this paper, we investigate the social and technical factors that contribute to the acceptance and effectiveness of models and simulations, what we refer to as model confidence. We combine a literature review with practitioner interviews to identify constructs and attributes influencing model confidence. Model confidence results from the interplay of model-related, modeler-related and stakeholder-related constructs. The constructs identified in this study populate a model confidence framework currently being developed. They highlight important considerations for future research and practice to enable improved and increased use of models and simulations in multidisciplinary settings.