1
$\begingroup$

I've done my best to pore over the other posts on this site, and to the best of my ability I can't seem to find a good solution to my problem, so I apologize in advance if elements of my question are covered elsewhere. Basically I'm asking if there is a faster way to do what I'm trying to do. I feel like my current implementation is a bit clumsy and would like to know how it can be improved. Here's my situation:

I'm working on a population genetics simulation. You enter in some initial data, click to run the simulation, then it spits some graphs and stats back at you. The simulation tracks a population of 'creatures' over a specified number of years. Currently the simulation operates on two large lists that have roughly the same format:

{
  {birthYear, deathYear, geneticInfo...},
  {birthYear, deathYear, geneticInfo...},
  ...
}

birthYear is always an integer, but can be negative. deathYear is always a positive integer OR Null if the creature is alive. geneticInfo is a placeholder I'm using here to represent series of values (currently True|False, but I've considered making them integers) that describe different genes the creatures have. Notably, geneticInfo is itself not a list, just a stand in for the arbitrary number of values I may use there so Dimensions[list] is {n, (2 + # of genes)}.

The bookOfLiving list holds only those creatures who were alive at the beginning of the year. The bookOfDead list is a running tally of all creatures that have died during the course of the simulation. In practice I've found this to be faster than keeping them all in the same list, since as the years go by both lists can get quite large and therefore slow to manipulate. Of course, I'm open to changing that if something else will work better. Here is an abbreviated version of the code for the simulation:

While[yrsDone < yrsMax,
  yrsDone++;

  (* First determine which living creatures died this year *)
  (* The result of liveOrDie[] returns the original list of a creatures properties, *)
  (* but will replace the deathYear with an integer if they have died before breeding*)
  bookOfLiving = Map[liveOrDie[#, yrsDone, simParameters] &, bookOfLiving];

  (* Now select those whose deathYear is an integer and add them to bookOfDead *)
  bookOfDead = Join[bookOfDead, Select[bookOfLiving, IntegerQ[#[[2]]] &]];

  (* Now rewrite bookOfLiving using only those whose deathYear is not an integer *)
  bookOfLiving= Select[bookOfLiving, (!IntegerQ[#[[2]]]) &];

  (* Now determine breeding population, number of new births, and rewrite bookOfLiving, if needed *)
  bookOfLiving = With[
    {breedPop = Select[bookOfLiving, ((breedRange[[1]] <= ageNow[yrsDone, #[[1]]] <= 
      breedRange[[2]]) && (#[[2]] === Null)) &]},

      If[Length[breedPop] >= 1,
        Join[bookOfLiving, breed[breedPop, yrsDone, simParameters]], bookOfLiving]
];

A few notes about some other functions and variables that may help:

simParameters, similar to geneticInfo, is a placeholder for the other information that I feed into some functions and not a list (though it may contain some [typically very short] lists).

liveOrDie[] calls several functions to determine, given a creatures attributes and the simulation's parameters, if a creature has died that year and returns the creature's original information, but with an integer deathYear if the creature died. Because it may have to return Null for the death year it is currently not compiled, but the function it calls to determine the probability of death is compiled. This is probably the biggest area where slowdowns occur because I couldn't come up with a batter strategy for indicating that a death occurred without rewriting individual values in the bookOfLiving.

ageNow[] calculates a creatures' current age based on its birth year and the current year. It is compiled, but I've considered using memoization instead. For a sufficiently large population, I figure the time savings may be non-negligible (there may be thousands alive at any one time). I prefer to use the birth and death year so that when the simulation is over I can calculate what the population looked like at any given time point instead having to calculate statistics each year and save them in another form. I'm open to change on this strategy if there is a good argument for it.

breedRange is a pair of values defining the minimum and maximum ages at which creatures may breed. It is set before the simulation is run, so it does not change during execution. The two values are always integers greater than or equal to 0.

For the moment my biggest bottleneck is coming from the list reassignments and repeated selections, but I can't think of a good way to simplify or enhance the efficiency of these steps. I have tried to keep as many of the functions as possible compiled to save time, but this is difficult given some of the restrictions on arguments to compiled functions. Would it be worth looking into any of the new FunctionCompile features as compared to Compile? For what it's worth I've double checked that none of my compiled functions bail out of compiled execution for some reason, so that should not be a source of any slowdown. Another awkward note is that the simulation actually takes about 50% longer to run when I used ParallelMap instead of Map for determining which creatures were dead each iteration even though I launched the kernels ahead of time. Though, it is harder to compare because, as written, it appears that the kernels don't use the same RNG, because my results will differ with each run even when I set a seed in advance. Any help or advice people can provide would be greatly appreciated! I'll provide additional details if necessary, but I tried to simplify the code I provided as much as possible to make it understandable.

$\endgroup$
10
  • 3
    $\begingroup$ Gosh, the effort in the post is appreciated but it truly is hard to comment without a working example. Do you have a github repo or anything? $\endgroup$
    – ktm
    Commented Apr 1, 2020 at 20:55
  • $\begingroup$ @user6014 thanks for the kind words. Sadly, I do not have a repo (this is just a project for my own amusement, so a single notebook works for me). $\endgroup$
    – Texas Red
    Commented Apr 1, 2020 at 21:04
  • $\begingroup$ If you want to upload a notebook/example dataset which demonstrates the speed issues I'd take a look (no promises it helps, but I would look :) ) $\endgroup$
    – ktm
    Commented Apr 1, 2020 at 21:05
  • 1
    $\begingroup$ "I've done my best to pour over the other posts on this site..." - that explains the wet feeling. $\endgroup$
    – ciao
    Commented Apr 1, 2020 at 22:24
  • $\begingroup$ @ciao Haha, this is literally the first time I've ever had to spell that instead of speak it! You learn something every day. $\endgroup$
    – Texas Red
    Commented Apr 1, 2020 at 22:52

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.