3
$\begingroup$

I have spent some time trying to figure this out.

I have a function that appears to be the reason my program is freezing. I am using both MemoryConstrained[] and TimeConstrained[], however, it seems that they are not preventing the function from freezing. Is there something I am overlooking with the following code?

getResultsForPop[population_, randomData_, bounds_, 
  maxBits_, maxTimeWaitPerFunc_, onlyGains_] := Module[{outcomes},
  outcomes = ParallelTable[
    Quiet[
     MemoryConstrained[
      TimeConstrained[
       findResultsOfCreature[population[[z]], randomData, 
        bounds, maxBits, onlyGains], maxTimeWaitPerFunc], maxBits]]
    , {z, Length[population]}]
  ]

If not, is there anything else that could be causing my program to freeze? For more description, the program runs for a little while (in a Table[] function loop), and then, after going through a few iterations of Table[], stops, at the function described above.

If anyone has any ideas for how I could get this to stop freezing, or just know how to better debug this function, that would help a bunch - thanks!

Also, if anyone would like me to post more code to better demonstrate the problem, just let me know. Thanks!

Edit: Here are examples of input variables:

population[[1]] = {<|p10 -> -0.594626, p11 -> 0.228192|>, {{NevilleThetaN[0.305105 p11, 
    0.376564 p11], {{p10, -0.16783}, {p11, 0.0934782}, {p11, 
     0.376342}, {p11, 0.450423}}}}, {p11, p10, p10, p10, p10}, {p10, 
  p11, p11}}

population consists of a list of lists that are of a similar structure to the one above, and the length of population is usually 600.

randomData[[1]] = {"2012-02-06 00:00:00-05:00", {28.45, 28.47, 27.77, 28.2, 2575725}}

Is the form for randomData, and it is usually of length 100.

bounds = 100
maxBits = 500000
maxTimeWaitPerFunc = 0.5
onlyGains = True

Also, to clarify, seeing as how this will better explain the question, randomData is not actually randomly generated data, but rather 100 bars of stock data that were taken starting at a random day.

The function findResultOfCreature[] takes a given member of the population (population), where we can call it creature, like the one shown above, as well as the list of stock data (randomData), where each element in this list represents one day of stock data (including the open, close, high, low, and volume) and feeds in the stock data day by day into the creature, and the creature gives an output relative to the input. The output is then used to generate a list of buys and sells, and create a running total of what percentage is made by following the output of creature, where 1 would mean no positive or negative percentage returns, and anything above 1 is positive, and anything below one represents negative returns.

 findResultsOfCreature[population[[z]], randomData, 
    bounds, maxBits, onlyGains]

could return something like the following, for example:

{1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.,1.01869,1.01869,1.01869,1.01869,1.01869,1.01869,1.01869,1.01869,1.01869,1.01869,1.05873,1.05873,1.05873,1.05873,1.05873,1.05873,1.05873,1.05873,1.05873,1.05873,1.04858,1.04858,1.04858,1.04858,1.04858,1.04858,1.04858,1.04858,0.979256,0.979256,0.979256,0.979256,0.979256,0.979256,0.979256,0.979256,0.979256,0.979256,0.914604,0.914604,0.914604,0.914604,0.914604,0.914604,0.914604,0.914604,0.914604,0.914604,0.908085,0.908085,0.908085,0.908085,0.908085,0.902738,0.902738,0.902738,0.902738,0.902738,0.902738,0.902738,0.902738,0.902738,0.902738,0.902738,0.902738,0.902738,0.902738,0.89815,0.89815,0.89815,0.89815,0.89815,0.89815,0.917503,0.917503,0.917503,0.917503,0.917503,0.917503,0.917503,0.932286,0.932286,0.932286,0.932286,0.899293}

Hopefully this helps with clarity.

To sum it up, however, what is confusing me the most is that, regardless of what is happening in the functions that are being called inside of getResultsForPop[], shouldn't MemoryConstrained[] and TimeConstrained[] be enough to prevent the program from freezing?

Edit: After thinking a little about this, could it be possible that none of the individual components of the ParallelTable[] are exceeding the memory limit, but that all of them, when put together, are doing so, and causing the program to freeze? I will have to check and see if compensating for this helps the program to stop freezing.

$\endgroup$
  • $\begingroup$ Please also provide the input that makes this function freeze. $\endgroup$ – Henrik Schumacher Oct 25 '18 at 21:43
  • $\begingroup$ Alright. So, I can provide an example of the input, but the population variable is very large and would fill pages and pages. Would a portion of this data (where population is a list of similar items) suffice? $\endgroup$ – Jmeeks29ig Oct 25 '18 at 21:55
  • $\begingroup$ Yes, a portion of the list can already help to learn about the shape of data. Also helpful would be the absolute size of the dataset. But maybe you can also provide a line of code to produce random data? SeedRandom can help to make a "random" dataset reproducible. Oh, and of course, we need to know what findResultsOfCreature is. $\endgroup$ – Henrik Schumacher Oct 25 '18 at 21:59
  • $\begingroup$ Alright, I made the edit. There is still a lot more to the program, but hopefully that is close enough information - I appreciate the help! $\endgroup$ – Jmeeks29ig Oct 25 '18 at 22:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.