A large number of studies have pointed out the role of working memory throughout numerical development. Working memory capacities seem to be improved after training and some studies have observed an impact of working memory training on academic performance. In our study, we examined whether training visuo-spatial working memory (with Cogmed) enhances working memory abilities and numerical development in the short and middle term in 5-6 year-old children. Fourty six children were randomly assigned to the experimental condition (adaptive working memory training) or the control condition (non-adaptive, demo version). The program was implemented daily for a period of five weeks in both groups. We observed an immediate impact of the adaptive version on visuo-spatial sketchpad and visuo-spatial central executive abilities and a small impact on Arabic number comparison. No training effect was observed in verbal working memory, in counting, collection comparison and addition. Furthermore, the observed effects were not sustained ten weeks later. These results are discussed in the context of specific and general cognitive factors that support numerical development and we argue against the idea of developing general cognitive factors to efficiently boost numerical development.