I'm currently trying to create some test data to check a numerical implementation against Mathematica's implementation of some functions.
In order to do this, I've been generating a broad range of arguments with a (Parallel)Table which works fine for a function with a few arguments, but I have one function I want to test which takes 4 integer arguments and 10 floating point arguments:
f[n1, n2, n3, n4, x0, x1, x2, x3, x4, x5, x6, x7, x8, x9]
The thing is creating a broad range of values use a table results in a simply enormous amount of data which cannot be held in memory. For example even:
realSample[n_] := Join[-10^Subdivide[-10, 10, n], 10^Subdivide[-10, 10, n], Subdivide[-10, 10, n]];
Table[{n1, n2, n3, n4, a, b, c, d, e, f, w, x, y, z},
{n1, 0, 1}, {n2, 0, 1}, {n3, 0, 1}, {n4, 0, 1},
{x0, realSample[1]}, ...
]
results in something that is just unmanageable even for n = 1
simply because of the sheer number of arguments.
So instead, I'd like to take a random sample of the whole table (say a million); however, RandomSample
doesn't stream the table and the same problem as before still comes up.
Short of implementing a reservoir sampling, is there anything in Mathematica that allows me to take a random sample of the full table without loading the whole table into memory?