The main focus of this article is to provide a mathematical study of greedy algorithms for the construction of reduced bases so as to approximate a collection of parameter-dependent random variables. For each value of the parameter, the associated random variable belongs to some Hilbert space (say the space of square-integrable random variates for instance). But carrying out an exact greedy algorithm in this context would require the computation of exact expectations or variances of parameter-dependent random variates, which cannot be done in practice. Instead, expectations and variances can only be computed approximately via empirical means and empirical variances involving a finite number of Monte-Carlo samples. The aim of this work is precisely to study the effect of finite Monte-Carlo sampling on the theoretical properties of greedy algorithms. In particular, using concentration inequalities for the empirical measure in Wasserstein distance proved by Fournier and Guillin [Probab. Theory Related Fields 162 (2015), pp. 707–738], we provide sufficient conditions on the number of samples used for the computation of empirical variances at each iteration of the greedy procedure to guarantee that the resulting method algorithm is a weak greedy algorithm with high probability. Let us mention here that such an algorithm has initially been proposed by Boyaval and Lelièvre [Commun. Math. Sci. 8 (2010), pp. 735–762] with the aim to design a variance reduction technique for the computation of parameter-dependent expectations via the use of control variates constructed using a reduced basis paradigm. The theoretical results we prove here are not fully practical and we therefore propose a heuristic procedure to choose the number of Monte-Carlo samples at each iteration, inspired from this theoretical study, which provides satisfactory results on several numerical test cases.