In the literature on numerical cognition, the presence of the capacity to distinguish between numerosities by attending to the number of items, rather than continuous properties of stimuli that correlate with it, is commonly taken as sufficient indication of numerical abilities in cognitive agents. However, this literature does not take into account that there are non-numerical methods of assessing numerosity, which opens up the possibility that cognitive agents lacking numerical abilities may still be able to represent numerosity. In this paper, I distinguish between numerical and non-numerical methods of assessing numerosity and show that the most common models of the internal mechanisms of the so-called number sense rely on non-numerical methods, despite the claims of their proponents to the contrary. I conclude that, even if it is established that agents attend to numerosity, rather than continuous properties of stimuli correlated with it, an answer to the question of the existence of the number sense is still pending the investigation of a further issue, namely, whether the mechanisms the brain uses to assess numerosity qualify as numerical or non-numerical.