Assessing and Benchmarking Learning Outcomes of Robotics-Enabled STEM Education

Abstract

Experienced middle school mathematics and science teachers were recruited for a pilot study. The teachers separately responded to a survey related to determining expected learning outcomes based on their traditional teaching, classroom experiences and observations, and self-brainstorming. The teachers then received training on how to design, develop, and implement robotics-enabled lessons under a design-based research approach for experiential learning, and taught robotics-enabled lessons to a selected student population in classroom settings. The teachers then responded to the survey for the robotics-enabled teaching. For each case (traditional and robotics-enabled), the survey responses were analyzed, and a set of expected learning outcomes of math and science lessons was derived separately. The thematic analysis results showed that the expected learning outcomes for the robotics-enabled lessons were not only related to the educational gains (content knowledge) observed in traditional teaching, but also to the improvements in the behavioral, social, scientific, cognitive, and intellectual aptitudes of the students. Then, a set of metrics and methods were proposed for assessing the learning outcomes separately. To validate the assessment metrics and methods, teachers from different schools taught two selected robotics-enabled lessons (one math, one science) to same grade students, and separately assessed the learning outcomes of each student using the proposed metrics and methods. The learning outcomes were then compared and benchmarked between schools and subjects. The results of a user study with the teachers showed user acceptance, effectiveness, and suitability of the assessment metrics and methods. The proposed scheme of assessing learning outcomes can be used to assess and justify the benefits and advantages of robotics-enabled STEM education, benchmark the outcomes, help improve teaching preparations, motivate decision-makers to confer on robotics-enabled STEM education and curricula development, and promote robotics-enabled STEM education.