Sound symbolism is the phenomenon by which certain kinds of phonemes are associated with perceptual and/or semantic properties. In this paper we explored size sound symbolism (i.e., the mil/mal effect) in which high-front vowels (e.g., /i/) show an association with smallness, while low-back vowels (e.g., /ɑ/) show an association with largeness. This has previously been demonstrated with nonwords, but its impact on the processing of real language is unknown. We investigated this using a size judgment task, in which participants classified words for small or large objects, containing a small- or large-associated vowel, based on their size. Words were presented auditorily in Experiment 1 and visually in Experiment 2. We did not observe an effect of vowel congruence (i.e., between object size and the size association of its vowel) in either of the experiments. This suggests that there are limits to the impact of sound symbolism on the processing of real language.