A Randomized Study to Assess the Effect of Including the Graduate Record Examinations Results on Reviewer Scores for Underrepresented Minorities

Abstract

Abstract Whether requiring Graduate Record Examinations (GRE) results for doctoral applicants affects the diversity of admitted cohorts remains uncertain. This study randomized applications to 2 population-health doctoral programs at the University of California San Francisco to assess whether masking reviewers to applicant GRE results differentially affects reviewers’ scores for underrepresented minority (URM) applicants from 2018–2020. Applications with GRE results and those without were randomly assigned to reviewers to designate scores for each copy (1–10, 1 being best). URM was defined as self-identification as African American/Black, Filipino, Hmong, Vietnamese, Hispanic/Latinx, Native American/Alaska Native, or Native Hawaiian/Other Pacific Islander. We used linear mixed models with random effects for the applicant and fixed effects for each reviewer to evaluate the effect of masking the GRE results on the overall application score and whether this effect differed by URM status. Reviewer scores did not significantly differ for unmasked versus masked applications among non-URM applicants (β = 0.15; 95% CI: −0.03, 0.33) or URM applicants (β = 0.02, 95% CI: −0.49, 0.54). We did not find evidence that removing GREs differentially affected URM compared with non-URM students (β for interaction = −0.13, 95% CI: −0.55, 0.29). Within these doctoral programs, results indicate that GRE scores neither harm nor help URM applicants.