This study focuses on variation of the prevalence rate of COVID-19 over time by age and race/ethnicity, and how neighborhood social vulnerability affects the COVID-19 prevalence in the whole epidemic as well as its three consecutive sub-waves in Milwaukee County, Wisconsin. It found that the highest prevalence rate was for young adults (18-44). Hispanics and Asians were more likely to be infected than were non-Hispanic whites and African Americans. The high neighborhood social vulnerability was associated with greater risk of infection especially for persons over age 25, for Hispanics and Asians. High prevalence rates were significantly and strongly associated with all major factors of the social vulnerability in early stage of the pandemic, especially with the factor associated with Hispanic and immigrant population. Throughout the epidemic, the Hispanic/immigrant and African American factors had a reduced but still significant effect, but the socioeconomic factor was not significant and the explained variance across neighborhoods was smaller.