3
$\begingroup$

I believe JoinAcross may have a memory leak but would appreciate some help looking at it. It appears to be small when used just once but when iterated a lot it eventually crashes my wolframscripts. I've tried Share[] and ClearSystemCache[] but they don't fix the issue. I've written the below toy example to show the leak:

SeedRandom[1234];
data1 = Table[<|"Date" -> DatePlus[Today, t],"Value" -> RandomReal[]|>, {t,1, 10000}];
data2 = Table[<|"Date" -> DatePlus[Today, t],"Value" -> RandomReal[]|>, {t,1, 10000}];
Do[
  memoryBefore = MemoryInUse[];
  data = JoinAcross[data1, data2, "Date", "Inner", KeyCollisionFunction -> ({"Data1" <> #, "Data2" <> #} &)];
  ClearAll[data];
  memoryAfter = MemoryInUse[];
  memoryChange = memoryAfter - memoryBefore;
  memoryInMegabytes = N[memoryChange/2^20];
  WriteLine[$Output, "Memory Change (MB): " <> ToString@memoryInMegabytes];
  ,
  {10}
];

This gives the output:

(*
Memory Change (MB): 7.17958
Memory Change (MB): 4.73375
Memory Change (MB): 4.73399
Memory Change (MB): 4.7343
Memory Change (MB): 4.7346
Memory Change (MB): 4.7346
Memory Change (MB): 4.7346
Memory Change (MB): 4.7346
Memory Change (MB): 4.7346
Memory Change (MB): 4.73454
*)

Thank you for any help!

$\endgroup$
5
  • $\begingroup$ See How to report bugs?. $\endgroup$
    – Jason B.
    Apr 11 at 11:42
  • 1
    $\begingroup$ $HistoryLength = 0 will give some remedy. However, a small increase is still visible. $\endgroup$ Apr 11 at 14:52
  • $\begingroup$ Yeah, I see that too. In my production scripts I've put $HistoryLength = 0 and it gives me more time but still does eventually crash. $\endgroup$
    – rfrasier
    Apr 12 at 2:34
  • $\begingroup$ Anyway, I have reported the bug to Wolfram though I can't say for sure if it is indeed a bug. $\endgroup$
    – rfrasier
    Apr 12 at 2:45
  • $\begingroup$ Received this response from Wolfram: It does appear that JoinAcross is not behaving properly. I have forwarded an issue report to our developers with the information you provided, and added your contact information to the report so that you can be notified when it is resolved. $\endgroup$
    – rfrasier
    Apr 14 at 8:08

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.