7
$\begingroup$

I am coding stuff manipulating indirection arrays and I have some code like:

createDupInvIndir[indirection_?VectorQ,duplicate_?VectorQ]:=
    Block[{n,dupInvIndir},
          n=Length[indirection];
          Assert[n==Length[duplicate]];
          dupInvIndir=ConstantArray[0,n];

          For[i=1,i<=n,i++,
             dupInvIndir[[indirection[[i]]]]=duplicate[[i]];
          ];

          Return[dupInvIndir];
    ];

We agree that this is not a good functional / Mathematica coding style. However, for the moment I have no idea to get an elegant/efficient way to remove the loop (using functions like Map, Scan...). Any suggestion/idea?


to check it works:

ind={3,5,4,2,1,6}
dup={1,2,2,3,3,3}
createDupInvIndir[ind,dup]

Output:

{3,3,1,2,2,3}

More context:

  • indirection: is an indirection array got from SortBy function used with Range[1,n] to sort another array.
  • duplicate: count different successive elements to detect duplicates

The array dupInvIndir that satisfies the relation:

dupInvIndir[[indirection[[i]]]]=duplicate[[i]]

is used to get the positions (taking into account the duplicate) of the sorted data without explictely reordering the data.

Here is a complete working example:

createDupInvIndir[indirection_?VectorQ,duplicate_?VectorQ]:=
    Block[{n,dupInvIndir},
          n=Length[indirection];
          Assert[n==Length[duplicate]];
          dupInvIndir=ConstantArray[0,n];
          For[i=1,i<=n,i++,
          dupInvIndir[[indirection[[i]]]]=duplicate[[i]];
          ];
          Return[dupInvIndir];
    ];

createDuplicate[data_List,indirection_?VectorQ]:=
Block[{dup},
      dup=Tally[Range[Length[indirection]],(data[[indirection[[#1]]]]==data[[indirection[[#2]]]])&];
      dup=Flatten[MapThread[ConstantArray[#1,#2]&,{Range[Length[dup]],Part[dup,All,2]}]];
      Return[dup];
];

data={{3,4},{3,5},{1,2},{1,2},{2,3},{3,6},{5,6}} 
indirection=SortBy[Range[Length[data]],data[[#]]&]   (* {3,4,5,1,2,6,7} *)
duplicate=createDuplicate[data,indirection]          (* {1,1,2,3,4,5,6} *)
dupInvIndir=createDupInvIndir[indirection,duplicate] (* {3,4,1,1,2,5,6} <- Final result *)

(* same stuff with *explicit* sort: data components are moved *)
DeleteDuplicates[Sort[data]] (* {{1,2},{2,3},{3,4},{3,5},{3,6},{5,6}} *)

Take the first element of dupInvIndir, 3, it means that the first element of the (implicitely) sorted data array is data[[3]], ( -> {1,2} ). This can be compared to the explicit DeleteDuplicate[Sort[data]]

$\endgroup$
  • 3
    $\begingroup$ It would be somewhat easier with some context on what you actually try to achieve there... $\endgroup$ – Henrik Schumacher Jan 31 at 11:22
  • $\begingroup$ @HenrikSchumacher please holds on, I will add more context $\endgroup$ – Picaud Vincent Jan 31 at 11:32
12
$\begingroup$

Apparently, you try to apply a permutation given by list indirection to a vector duplicate.

Here are several ways to do it, each along with its timing:

n = 1000000;
indirection = RandomSample[Range[n], n];
duplicate = RandomInteger[{-n, n}, n];


First@RepeatedTiming[
  result0 = createDupInvIndir[indirection, duplicate];
  ]

First@RepeatedTiming[
  result1 = ConstantArray[0, Length[duplicate]];
  result1[[indirection]] = duplicate;
  ]

First@RepeatedTiming[
  result2 = 
    Normal@SparseArray[Partition[indirection, 1] -> duplicate,Length[duplicate]];
  ]

First@RepeatedTiming[
  result3 = duplicate[[InversePermutation@indirection]];
  ]

result0 == result1 == result2 == result3

1.799

0.012

0.016

0.015

True

This is another one of the many examples that highlights why For should not be used (in uncompiled code).

$\endgroup$
  • $\begingroup$ Thanks for this quick answer, I already had the Normal @ SparseArray solution in mind but I eliminated it because I had the feeling it was not optimal concerning efficiency (a lot of manupilation, rules, array etc..). I did not know about the ToPackedArray package, thanks! (and yes indirection is a permutation of [1..Length[duplicate]]) $\endgroup$ – Picaud Vincent Jan 31 at 11:37
  • $\begingroup$ The benchmark you just added confirms my feeling about perfs. Thanks a lot $\endgroup$ – Picaud Vincent Jan 31 at 11:39
  • 1
    $\begingroup$ You're welcome. Developer`ToPackedArray is useful if you handle unpacked arrays that could be packed (with pure machine integers or machine precision numbers (real or complex)). Of course, it is always better never to produce any unpacked arrays. Developer`PackedArrayQ lets you check whether an array is packed or not. I removed the use of Developer`ToPackedArray in my answer since RandomSample and RandomInteger are guaranteed to produce packed arrays (if the occurring integers are not too large). $\endgroup$ – Henrik Schumacher Jan 31 at 11:41
  • 1
    $\begingroup$ For and the like can be quite useful. They are no substitute when alternatives exist that are vectorized though. $\endgroup$ – Daniel Lichtblau Jan 31 at 16:20
  • $\begingroup$ It might be worth nothing that replacing the For loop with a Do loop directly gives you a 25% speedup, without doing anything fancy. Also, it seems you've accidentally deleted the definition of ToPack from your answer... $\endgroup$ – Lukas Lang Jan 31 at 21:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.