Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

This is a two part question, and for that I apologize.. but they're related!

Here's what I'm working with:

d1 = Import["file.CSV", "List"]

size = Length[d1]

dis1 = RandomChoice[{d1}, {100, size}]
  • Q1: Length views d1 as $300,000$ individual elements but RandomChoice views it as a single element. Thus when I execute I get $100$ repetitions of d1 in the exact order d1 is presented in.

    Have I made an annotation error in assigning the RandomChoice number pool as {d1}? If I replace {d1} with a hand written list of values it executes perfectly, so I assume it is a presentation issue.. but I can't tell what I've done wrong from the documentation of the function.

  • Q2: Asking to make $100$ random lists of size ~ $300,000$ elements requires a lot of memory. The thing is I don't need to save each RandomChoice output, I just need the mean/median/SD/SEM for each of the $100$ sets I've tasked RandomChoice with.

    Is there a way to tell the program to spit out those end-point values and dump the accumulated list after each one is generated?

share|improve this question

2 Answers 2

Q1: You don't need the curly-braces around d1 in the RandomChoice: That turns it into a list with one element - d1.

Q2: If memory utilization is more important than speed (because it is usually far better to generate samples/variates/etc. en masse), you can do something like this:

myBigList = RandomInteger[100, 20000];
listLen = Length[myBigList];

results = 
 Reap[Do[picks = RandomChoice[myBigList, listLen]; 
    Sow[{Mean[#], Median[#], StandardDeviation[#]} &@picks], {100}]][[2,
    1]]

(Replace my RandomInteger with your import).

share|improve this answer
    
Aha! Thank you very much for both answers. The Reap/Sow usage is excellent. And yes, memory is more important currently. I'll be iterating this procedure quite a lot. –  Justin Jan 29 at 5:39
    
A follow up query.. if anyone has input. Let's say instead of Sowing the Mean/Median/SD I directly take #1 and #2 lists from "picks" and perform a T-test on those and punch out the p-value, then repeat for the next 2 randomchoice sets. Is that possible? I didn't realize Mathematica would compute p-values for whole data sets. –  Justin Jan 29 at 6:07
    
@justin: Sure. Just replace listLen in the RandomChoice with {2,listLen}, and the whole {Mean[#], Median[#], StandardDeviation[#]} & part with TTest and you're set. Note that you'll get results for pairs of sampled lists, so adjust the iterator if needed. –  rasher Jan 29 at 6:36
    
Thanks. I appreciate it! I gave that a whirl but was coming up with very large p-values. I suspect the default of TTest is to compare the query set to 0. I made an attempt to recode as follows: results = Reap[Do[pick1 = RandomChoice[d1, {1, listLen}]; pick2 = RandomChoice[d1, {1, listLen}]; Sow[{TTest[pick1, pick2 , "PValue"]}], {1}]][[2, 1]] The goal being to generate the first list as the query set, and the second list as the comparison set. But it keeps telling me my array is the wrong size/shape. Thanks for the input folks! –  Justin Jan 29 at 6:49
    
@Justin: take the {1,listLen} to just listLen - the way you have it written, you're asking for a list of one element that is a list of listLen choices. You'll also then need to bracket the pick1 and pick2 in the TTest. There's no real need to pick these separately though, since TTest does not "favor" either list - it simply compares if the means are the same, so doing it as I showed is (a bit) faster. –  rasher Jan 29 at 7:03

For Q1, your syntax mistake is an extra pair of brackets. The following code should work properly:

d1 = Import["file.CSV", "List"]
size = Length[d1]
dis1 = RandomChoice[d1, {100, size}]
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.