1
$\begingroup$

Suppose I have the following:

Table[x[i,j] = f[i,j], {i,1,10},{j,1,10}];

I want to set variables x[i,j] to have value f[i,j] where f can be anything. The Table (and you could also achieve this with a Map and I'm sure many other ways) produces an undesirable list which will be discarded. This is a waste of memory, especially if f produces some large object, and the copies will have to be garbage collected even though I've suppressed the output.

To avoid this behaviour for lists I would normally do a Scan like this example in the documentation:

test = Scan[(u[#] = x) &, {55, 11, 77, 88}]

It sets u[55], u[11], u[77], u[88] without creating any intermediate lists and test is Null.

Compare this with a Map where test is populated with the values on the RHS of the Set. Also note this test return value is for illustrating the idea only and I expect to discard it:

test = Map[Set[u[#],x] &, {55, 11, 77, 88}];

Question:

  1. How can I use Scan over a multidimensional list such as Tuples[Range[10],2] to achieve the same effect as the Table example at the top of this question? I am not interested in using a For loop and I am only interested in achieving this with Scan if that's possible.

  2. Are my fears about the discarded return values wasting memory and adding to garbage collection time justified if the object on the RHS of the Set is very large? Can Mathematica tell when a Table or Map is about to discard the list they build up and avoid creating one? I suspect not e.g try MaxMemoryUsed[Table[x, 300000];]

Update:

Scan appears to be consistently worse for memory usage, much to my surprise. Why? Surely it can't be because of the Range that must be constructed first, because MaxMemoryUsed[Range[10000]] is only 80376 bytes.

f[i_] := RandomReal[i, {64, 64}];
MaxMemoryUsed[Scan[(x[#] = f[#]) &, Range[10000]]]
ClearSystemCache[];
MaxMemoryUsed[Table[y[i] = f[i], {i, 10000}]]
ClearSystemCache[];
MaxMemoryUsed[Do[z[i] = f[i], {i, 10000}]]
ClearSystemCache[];
| improve this question | |
$\endgroup$
  • $\begingroup$ Note: a question with similar title appeared here mathematica.stackexchange.com/questions/154609/… but the OP didn't need a Scan to begin with and the answer doesn't even use a Scan so it's not that relevant. $\endgroup$ – flinty Jun 24 at 15:04
  • 2
    $\begingroup$ Do can be thought of a as a drop-in replacement for Table when the list is not necessary. $\endgroup$ – C. E. Jun 24 at 15:10
  • $\begingroup$ @C.E. I know I can do it with For, Do, While etc. $\endgroup$ – flinty Jun 24 at 15:11
  • $\begingroup$ I might be misunderstanding, but wouldn't something like this work? (u[##] = f[##]) & @@@ {{4, 4}, {5, 2}, {4, 3}} (The latter are just some random sample locants) $\endgroup$ – MarcoB Jun 24 at 15:16
  • $\begingroup$ @MarcoB no because test = (u[##] = f[##]) & @@@ {{4, 4}, {5, 2}, {4, 3}}; is not Null. It creates a list of results which needs to be garbage collected - which is fine if the items produced by f are small, but slow if they are large. This is not an XY problem, I'm only interested in how to do it with Scan - I already know several other ways to achieve the same effect as the Map or Table approach. $\endgroup$ – flinty Jun 24 at 15:27
3
$\begingroup$

I find Scan to be a strange solution to your problem since the input to Scan is a list of the same dimension that the output of Table would be, and a list like that is what you would like to avoid.

Here is how you would do it:

Scan[(x[Sequence @@ #] = f[Sequence @@ #]) &, Array[List, {10, 10}], {2}]

As I suggested in my comments, Do is a drop-in replacement for Table without the output, but I understand that this is not what you want.

| improve this answer | |
$\endgroup$
  • $\begingroup$ The input list isn't important because it's just small indices. The output of f is why I need Scan. If you imagine f produces a lot of output, then the Table[...]; will go and run all these f and set x for each one. But it will also return copies of those results of f in a big list which is discarded. Scan just returns Null and nothing was wasted. $\endgroup$ – flinty Jun 24 at 15:35
  • $\begingroup$ @flinty Alright. $\endgroup$ – C. E. Jun 24 at 15:42
  • $\begingroup$ Weirdly though, I'm profiling both with MaxMemoryUsed and the Table does better than Scan - maybe Mathematica knows when the Table isn't going to return its contents and doesn't bother creating a list that's going to be discarded anyway. I also tried with this Scan[Set[Evaluate[x @@ #], f @@ #] &, Tuples[Range[10], 2]] $\endgroup$ – flinty Jun 24 at 15:50
  • $\begingroup$ @flinty Try using MemoryInUse[] instead of MaxMemoryUsed[]. I have had trouble interpreting the results of the latter in the past $\endgroup$ – MarcoB Jun 24 at 16:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.