Comprehensive consideration of uncertainty in simulation use

Abstract

Drawing appropriate conclusions from simulation results requires a correct understanding of the accuracy and context of those results. Simulation communities often assess simulation results without considering fully uncertainties that might impact the accuracy and context of those results. This creates potential for inappropriate conclusions from simulation results. Much useful work has been done in uncertainty quantification, but most of those efforts have addressed uncertainty in particular parameters and areas. Unfortunately they have not addressed all areas of potential uncertainty that might impact simulation results. A paradigm exists that facilitates consideration of all potential sources of simulation uncertainty. This paper examines simulation uncertainties using that paradigm and indicates potential magnitude of uncertainties for simulation results in various areas. A comprehensive approach to simulation uncertainty not only reduces the likelihood of drawing inappropriate conclusions from simulation results, but it also provides information that can help determine where it is most useful to invest verification and validation resources in efforts to reduce uncertainty in simulation results (i.e., to improve the accuracy of simulation results). Comprehensive assessment of simulation uncertainty may have drawbacks. When addressed comprehensively, simulation uncertainty tends to be larger than desired, and those announcing such run the risk of being bearers of bad news. Realistic appreciation for the uncertainty associated with simulation results can also decrease the importance of those simulation results in decision processes. On the positive side, such realistic and comprehensive appreciation for simulation uncertainty provides a solid factual and logical basis for how to proceed, whether by improving simulation capabilities or by developing alternative approaches to support decision processes. A perspective from comprehensive consideration of simulation uncertainty helps to ensure a proper context for simulation results.