Development of the Water Instrument: a comprehensive measure of students’ knowledge of fundamental concepts in general chemistry

Abstract

General Chemistry serves virtually all STEM students. It has been accused of covering content in a “mile wide and inch deep” fashion. This has made it very difficult to assess, where chemistry educators have relied on assessments of specific topics. Assessing across all these different topics requires introducing many different chemical systems and contexts, which may pose a threat to validity in the measurement of students’ knowledge of general chemistry concepts. With many different systems and contexts, it is possible that students will have varying familiarity, may resort to memorization, or rely on fragments of knowledge to answer. To account for challenges which may arise with different systems and contexts, we have developed an assessment instrument for measuring students’ understanding of key concepts from a year-long sequence of general chemistry that relies on a single context: water. The Water Instrument was developed using exploratory sequential design to target eight of the ten anchoring concepts for general chemistry. Psychometric results will be presented from the final pilot administration, where an item response model was used to evaluate the assessment. Further evidence gathered through Response Process Validity interviews will be presented. The evidence gathered indicates this assessment offers a valid and reliable estimate of students’ understanding of general chemistry concepts in the context of water, which makes this instrument promising for general chemistry assessment. The comprehensive nature of the assessment can provide rich information to instructors regarding their students’ conceptual knowledge of the wide range of topics covered in a year-long sequence of general chemistry.