This question is possibly related to this topic, although I think I have something a little more specific in mind. Recently I've been working with dispatch tables for a project, since it seemed like a much more optimized way of implementing replacement rules. I haven't had many problems in actually creating the table and saving it (it is roughly ~1 GB of a plain text file).

However, I'd now like to use the dispatch table in a different notebook. It appears that just calling Get["myDispatch"] is extremely slow and eats up memory. I've experimented with using .mx files created by DumpSave instead, but to no avail (the files created are roughly of the same size, and still crash my computer trying to load).

Is it possible to efficiently load a dispatch table, or should I think about other ways to store my replacement rules?

  • $\begingroup$ What makes it so large? Are the rules themselves large expressions or is it a huge number of simple rules? $\endgroup$ May 17, 2016 at 20:06
  • $\begingroup$ I would say it's mostly a huge number of simple rules. They take the form x[{n1,n2,...,n10}]-> a number, but there are lots of rules because there's essentially a rule for each permutation of the list {n1,n2,...,n10}. I'm simplifying it a bit, but the number of associations grows combinatorially with the size of the lists, and I'm considering many lists of different length. $\endgroup$ May 17, 2016 at 20:35
  • $\begingroup$ What happens if you use Export[“some/path“,Compress@mydispatch,“String“] and use the appropriate Uncompress@Import[...] to load it? $\endgroup$
    – Lukas
    May 17, 2016 at 21:38
  • $\begingroup$ Thanks for the suggestion - that does seem a little faster than before. I actually ended up getting the most mileage by just not memoizing values that I didn't need, following this helpful answer, if anyone ever stumbles upon this question in the future. $\endgroup$ May 21, 2016 at 2:05


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.