Strive for Measurement, Set New Standards, and Try Not to Be Evil

Abstract

I consider recent attempts to establish standards, principles, and goals for artificial intelligence (AI) through the lens of educational measurement. Distinctions are made between generative AI and AI-adjacent methods and applications of AI in formative versus summative assessment contexts. While expressing optimism about its possibilities, I caution that the examples of truly generative AI in educational testing have the potential to be overexaggerated, that efforts to establish standards for AI should complement the Standards for Educational and Psychological Testing and focus attention on the issues of fairness and social responsibility, and that scientific advance and transparency in the development and application of AI in educational assessment may be incompatible with the competitive marketplace that is funding this development.