Replacing pairs of values under constraints in a large nested list

Question

I have a list of integers that looks e.g.

t = {{{1170, 4150}, {1883, 7089}, {2105, 7704}}, {{2413, 8086}, {1928,
     7764}, {2126, 8312}}, {{3791, 10052}, {9832, 21949}, {10367, 
    23078}}}

I want to replace those value pairs that fall into a certain range (e.g. first element between 1000 and 2000 and second element between 3000 and 8000). The entire element should be replaced by e.g. Red. With the just mentioned criteria, the result should be:

t = {{Red, Red, {2105, 7704}}, {{2413, 8086},Red, {2126, 8312}}, {{3791, 10052}, {9832, 21949}, {10367,23078}}}

Now, the actual list is massive, thousands to millions of entries – it's an image. I've used multiple attempts and either failed (when trying using ReplacePart), or the code takes ages (when using If conditions).

Answer 1

t /. {a_ /; 1000 < a < 2000, b_ /; 3000 < b < 8000} -> Red

{{RGBColor[1, 0, 0], RGBColor[1, 0, 0], {2105, 7704}}, {{2413, 8086}, 
  RGBColor[1, 0, 0], {2126, 8312}}, {{3791, 10052}, {9832, 21949}, {10367, 23078}}}

which is the way it outputs on my machine because Red is a built-in symbol.

Answer 2

You my use ParallelMap. Read the Parallel Evaluation tutorial.

Using

t = RandomInteger[{500, 10000}, {100000, 3, 2}];

With

foo =
 If[
   Between[{1000, 2000}]@First@# && Between[{3000, 8000}]@Last@#,
   Red,
   #] &;

Then

new = ParallelMap[foo, t, {2}];

Will fill new with the replaced data having performed the replacement in parallel over the launched kernels. Number of kernels depends on your configuration and license.

Hope this helps.

Answer 3

This question is a more complex version of Replace values which obey certain criteria and the same principles apply: if you want a fast operation look for vectorized numeric approaches.

Since you want to operate on the first element of each pair separately from the second element we may want to separate those first using Transpose:

{t1, t2} = Transpose[t, {2, 3, 1}];

Since we are looking for values in a certain range method from SO 6026827 can apply.

Since performance is the goal I shall use explicit Subtract; re: (40927)

between = UnitStep[Subtract[#, #2]*Subtract[#3, #]] &;

This function produces a binary (zero or one) result and it is inherently Listable.

Therefore our replacement mask can be produced with Times taking the role of And:

mask = between[t1, 1000, 2000] * between[t2, 3000, 8000]

{{1, 1, 0}, {0, 1, 0}, {0, 0, 0}}

The SparseArray property "AdjacencyLists" is a fast way to get the positions 1 in each row:

pos = SparseArray[mask]["AdjacencyLists"]

{{1, 2}, {2}, {}}

Revision: an analysis of the first function I posted revealed that ReplacePart was a performance bottleneck, therefore I am replacing it with a Part assignment.

Row by row this can be used for replacement like this:

row = t[[1]];
row[[{1, 2}]] = Red;
row

{RGBColor[1, 0, 0], RGBColor[1, 0, 0], {2105, 7704}}

As a self-contained function:

f2[t_, new_, {lo1_, hi1_}, {lo2_, hi2_}] :=
   Module[{t1, t2, between, mask, pos, row},
     {t1, t2} = Transpose[t, {2, 3, 1}];
     between = UnitStep[Subtract[#, #2]*Subtract[#3, #]] &;
     mask = between[t1, lo1, hi1]*between[t2, lo2, hi2];
     pos = SparseArray[mask]["AdjacencyLists"];
     MapThread[(row = #; row[[#2]] = new; row) &, {t, pos}]
   ];

f2[t, Red, {1000, 2000}, {3000, 8000}]

{{RGBColor[1, 0, 0], RGBColor[1, 0, 0], {2105, 7704}},
 {{2413, 8086}, RGBColor[1, 0, 0], {2126, 8312}},
 {{3791, 10052}, {9832, 21949}, {10367, 23078}}}

This is quite a bit faster than bobbym's admittedly simpler ReplaceAll code:

big = RandomInteger[9000, {3000, 3000, 2}];

f2[big, Red, {1000, 2000}, {3000, 8000}]; // RepeatedTiming

{1.745, Null}

big /. {a_ /; 1000 < a < 2000, b_ /; 3000 < b < 8000} -> Red; // RepeatedTiming

{9.03, Null}