Using Multilevel Models to Explore Predictors of High School Students’ Nonresponse in Experience Sampling Method (ESM) Studies

Abstract

This study uses multilevel generalized linear models to examine predictors of high school students’ nonresponse when using the experience sampling method (ESM), a form of momentary data collection that captures participants’ situational thoughts, feelings, and emotions. Because ESM approaches often seek to generalize and compare participants’ emotional states across days and times, it is important to understand how and when participants may miss response opportunities, and further to explore how this response bias may limit generalizability of findings. Results from this study, conducted in three mid-Michigan high schools in 2013–2014 with a sample of 141 students, indicate that time of day and day of week are significantly related to a given participant’s odds of nonresponse. Specifically, ESM “prompts” occurring after school and over the weekend had much higher odds of being missed by participants, even after controlling for other covariates such as race/ethnicity, gender, and person-level emotional trends. These findings demonstrate that day and time contextual factors are significantly related to odds of nonresponse, and researchers using these approaches to compare widely different time contexts should be mindful of possible generalizability concerns.