For testing purposes, I'd like to create very large amounts of Random Integers, let's say between $-1000$ and $1000$. I'm using the usual
RandomInteger[{-1000, 1000}, 5]
command to create $5$-tuples. On my machine (8GB of RAM), this process becomes quite slow once in the
Table[
RandomInteger[{-1000, 1000}, 5]
,
{100000000}]
(one hundred million) region. It takes 17 seconds of Timing[]
for this list to be made, and I haven't even had any calculations on it.
Does this mean I should restrain using lists till somewhat lower as this threshold, or is there any other way of handling huge lists? Does MMA input the whole list in RAM before doing calculations with it? Is it possible to handle huge lists without including the whole list in RAM, for speeding purposes?
I thought of dividing the list, perhaps exporting to different files (CSV for example), and then doing calculations with every file individually. This is probably the same as Partition[]
does (without exporting)?
Thanks for any more suggestions!
RandomInteger[{-1000, 1000}, {100000000, 5}]
is about 5 times faster. $\endgroup$ – Kuba♦ Nov 6 '13 at 6:14Table
for this. See @Kuba suggestion. $\endgroup$ – RunnyKine Nov 6 '13 at 6:15big-list
tag: funny. $\endgroup$ – wolfies Nov 6 '13 at 6:31