We explored the multi-dimensionality of mathematics and working memory (WM) by examining the differential relationships between different areas of mathematics with visual, spatial, and verbal WM. Previous research proposed that visuospatial WM is a unique predictor of mathematics, but neuroimaging and cognitive research suggest divisions within visuospatial WM. We created a new WM task to isolate visuospatial WM’s visual and spatial components and maintained consistent design across tasks and found that spatial WM predicted mathematics and visual WM did not. We also found that verbal WM predicted all mathematics areas included, while spatial WM was a unique predictor of numerical understanding and geometry, not arithmetic and estimation. These findings integrate previous neuroimaging, cognitive and educational psychology research and further our understanding of the relationship between WM and mathematics.