How large is the the set of elliptic curves up to isomorphism over the rationals? What is the dimension of the space of nonzero rationals modulo squares? We want to make sense of these questions. Given a field K and a concept of a set of "arithmetic objects," like isomorphism classes of elliptic curves, we will define a sampling space, which is a kind of parameter space for objects over K. A sampling space is not unique in general, nor does it always exist, but when it does it helps us determine the complexity of the objects in question. Sampling spaces can also be seen as alternatives to coarse moduli spaces.