1. The issue of validation of mathematical and computational models of nature touch on the very foundations ofscience. Thereasonforthesedeeplyrooted issues in science is the question of how can formal constructs (models) be tested by physical observation. The renowned 20th century philosophers of science Popper 18,19 and Carnap 20 laid the foundation for the present day concepts of validation. The first technical discipline that began to struggle with the methodology and terminology of verification and validation was the operations research (OR) community.21-56 In the OR activities, the complexity of the systems analyzed could be extraordinary, for example, industrial production models, industrial planning, marketing models, national and world economic models, and war fighting models. These complex models commonly involve a strong coupling of complex physical processes, human behavior, and computer controlled systems. For these complex systems and processes, fundamental conceptual issues immediately arise with regard to assessing credibility of the model and the resulting simulations. Indeed, the credibility of most of these models cannot be validated in anymeaningfulway.
2. The meaning andthe clarity of the terms verification and validation took a major step forward in 1994 with the definitions developed by the Defense Modeling and Simulation Office (DMSO) of the Department of Defense.I The DMSOdefinitions were formulated based on the foundational work of the OR community referenced above. In 1998, the AIAA Computational Fluid Dynamics Committee on Standards adopted the definitions of DMSO.16 The definition of verification given by the AIAA Guide for the Verification and Validation of Computational Fluid Dynamics Simulationsis:
3. In science and engineering, CFD was one of the first fields to seriously begin developing concepts for validation methodology and validation experiments 5-7, 9-11,13,58-96_Much of this early work dealt with issues such as fundamental methodology, terminology, development of the concepts and procedures for validation experiments, confidence inpredictionsbasedon validated simulations, and methods of incorporating validation into the engineering design process. Essentially all of this early work dealt with CFD for aircraft and reentry vehicle aerodynamics, gas turbine engines, and turbopumps. In parallel with the aerospace activities and the OR work mentioned above, there were significant efforts in validation methodology in the field of surface and subsurface water quality modeling and safety assessment of underground radioactive waste repositories. 97- l03This water quality work is significant for two reasons. First,it addresses validation for complex processes in the physical sciences where validation of models is extremely difficult, if not impossible. The reason for the difficulty is that one of the key elements in the modeling is extremely limited knowledge of underground transport and material properties. For situations such as this, one must deal with calibration orparameter estimation in models, prior to considering validation. Second, because of this difficulty these fields have adopted statistical methods of validation assessment. As will be discussed shortly, we believe CFD must also begin adopting statistical methods of validation. Examining the literature from these diverse disciplines in operations research, earth sciences, and CFD clearly shows that each discipline developed concepts and procedures essentially independently.
4. Afinal comment should be made concerning the nonuniformity of the usage and meaning for the terms verification and validation. It is still common in CFD for people to misuse terms, for example, one refers to verification when one means validation. There is, however, a fundamentally different meaning of the terms verification and validation in other fields that must be noted. In 1984, the Institute ofElectrical and Electronics Engineers (IEEE) defined verification as follows:104, I05 ''The process of evaluating the products of a software development phase to provide assurance that they meet the requirements defined forthem by the previous phase." IEEE defined validation as:104,105: "The process of testing a computer programandevaluating the results to ensure compliance with specific requirements." Comparing these definitions with the DMSO/AIAA definitions given previously, it is immediately clear they mean something completely different. IEEE definitions are entirely referential, i.e., thevalue of the definition is related to the specification of "requirements defined for them by the previous phase" and "compliance with specific requirements." The substance of the meaning must be provided in the specification of additional information. Because those requirementsarenot statedin the definition, thedefinition does not contributemuch to theintuitive understandingof verification andvalidation. These same IEEE definitions for verification and validation have been adopted by the software 3ualit 7 assurance and computer science communities,/ 6,10 the nuclear reactor safety community,108,10, and the International Organization for Standardization (ISO)l 10, 111 The IEEE definitions for V&V are pointed out for two reasons. First, these definitions provide a distinctively differentperspective towardthe entire issue of verification and validation. This perspective asserts thatbecause of the extreme variety of requirements for modeling and simulation, the requirements should be defined in a separatedocument for each application, not in the definition of validation. Second, the IEEE definitions are the more prevalent definitions used in engineering, and one must be aware of the potential confusion when the DMSO/AIAA definitions are used in mixed disciplines. The IEEE definitions are dominant because of the worldwide influence of this organization. As a result, we expect long-term ambiguity and confusion. 2.2 AIAA Guide
5. In 1992, the AIAA Computational Fluid Dynamic Committee on Standards began a project to formulate the basic terminology and methodology in the verification and validation of CFD simulations. After 6 years of discussion and debate, the project culminated in the publication of Guide for the Verification and Vali%ation of Computational Fluid Dynamics Simulations. I The Guide defines a number of key terms, discusses fundamental concepts, and specifies general procedures for conducting verification and validation in CFD. AIAA Standardsdocuments are segregated into three levels of the state of the art:guides, recommended practices. and standards. The V&V Guide is at the first level, reflecting the early stageofdevelopment of concepts and procedures in V&V. It is also the first standards document to be published by any engineering organization on the topic ofV&V. The American Society of Mechanical Engineers is in the early stages of forming a new standards committee and developing a similar document in the field of solid mechanics_112