We investigate ice templating of aqueous dispersions of polymer coated colloids and crosslinkers, at particle concentrations far below that required to form percolated monoliths. Freezing the aqueous dispersions forces the particles into close proximity to form clusters, that are held together as the polymer chains coating the particles are crosslinked. We observe that, with an increase in the particle concentration from about 106 to 108 particles per ml, there is a transition from isolated single particles to increasingly larger clusters. In this concentration range, most of the colloidal clusters formed are linear or sheet like particle aggregates. Remarkably, the cluster size distribution for clusters smaller than about 30 particles, as well as the size distribution of linear clusters, is only weakly dependent on the dispersion concentration in the range that we investigate. We demonstrate that the main features of cluster formation are captured by kinetic simulations that do not consider hydrodynamics or instabilities at the growing ice front due to particle concentration gradients. Thus, clustering of colloidal particles by ice templating dilute dispersions appears to be governed only by particle exclusion by the growing ice crystals that leads to their accumulation at ice crystal boundaries.