A Proposed Model for Simulation Validation Process Maturity

Abstract

This paper proposes a model of process maturity for simulation validation. The development of this model begins by recognizing validation as a process that generates information as its sole product and therefore resembles a systematic quest for truth. These characteristics distinguish the simulation validation process from other processes such as those for manufacturing or software engineering. This development then substitutes process objectivity and the properties of quality information for the difficult-to-measure qualities of truth. The properties of information quality are defined by the completeness and correctness of the information and the confidence that the information producer has that the information is complete and correct enough to serve a particular purpose. These parameters come by amalgamating the notions for information quality held by the process improvement and traditional scientific communities. The validation process takes validation criteria, the referent, the simulation's conceptual model, verification results of the intermediate development products, and the simulation results as input and produces evidence assessing the simulation's validity as output. The quality of the input information ultimately limits the quality of the evidence that the validation process produces. The proposed model of validation process maturity thus structures the validation process into six levels defined by the quality of the input information, the quality of its information products, and the objectivity of those process components that contribute to those products. At the first level, the users demand absolutely no validation of the models they apply. The second level consists entirely of face validation where the validation evidence depends entirely upon subjective sources of requirements, referent, and validity judgments. The next level improves the objectivity and quality of the representational requirements. The next two levels progressively improve the objectivity of the conceptual modeling and results validation component processes and the quality of the evidence they produce. In the final maturity level, the validation process automatically transforms informal user need statements into formal validation criteria and then applies mathematical techniques to prove conceptual model and simulation results validity. In this model, the objectivity and quality of the validation evidence increases as the level of simulation validation process maturity increases.