7
$\begingroup$

I have several lists that I would like to match and operate values among them. For simplicity assume I have these two lists:

l1 = Transpose[{Range[1980, 2010, 1], Range[31]}]
l2 = {{1990, 1,100}, {2000, 2,200}, {2010, 3,300}}

I would like to divide the third column of l2 by the second column in l1 matching the year in both lists.

I should obtain:

l3 = {{1990, 1, 100/11}, {2000, 2, 200/21}, {2010, 3, 300/31}}

How can I set a function to do that?

$\endgroup$

5 Answers 5

10
$\begingroup$

Lookup says it all... ;)

lookuptable = AssociationThread[l1[[All, 1]], l1[[All, 2]]];
l3 = l2;
l3[[All, 3]] /= Lookup[lookuptable, l2[[All, 1]]];
l3

{{1990, 1, 100/11}, {2000, 2, 200/21}, {2010, 3, 300/31}}

Remark on performance:

It may sound a bit over the top for this tiny usage example, but I had very intentionall written this exactly like it is now: using AssociationThread, with a dedicated copy operation, and without Map-like constructs. Here is why:

Suppose thatl1 is of length m and that l2 is of length n. A naive lookup would cost you m n comparisons. This gets slow for large m and n. If you have to do many lookups, then it pays off quite soon to build a dedicated search data structure (e.g., a binary tree) for l1. Then you can boil down the runtime cost to something like $O(|\log(m)| \, n)$. That's precisely what I meant to do by using AssociationThread[l1[[All, 1]], l1[[All, 2]]]: It generates an Association, whose constructor builds up precisely such a search data structure. Morever, I used this precise syntax (I could also have used AssociationThread@@Transpose[l1] in order to not unpack any arrays (which can also have some mildly negative effect on runtime).

The reason why I do not use Map-like constructs is that they are - albeit being backed up by jit-compilation - still way slower than an ordinary compiled loop written in C/C++. The creators of Lookup knew that, too, so they implemented an overload of Lookup that can look up many keys in the association all at once (by implementing the loop over the keys as a C/C++ loop). So we get the looping for free. Moreover, if the type of keys allows (we use simple arrays of base types like integers here), then Lookup picks a code branch that does not involve the pattern matcher. The pattern matcher is very handy, but its power comes at a high performance cost.

The creators of Lookup where also kind enough to return us a packed array, if possible! So in principle, you can proceed with your computations with packed arrays if those computations allow it. In the example here, this does not happen, because exact rational numbers cannot be stored in packed arrays. But supposing that l3 and Values[lookuptable] where packed arrays of machine precision reals, then after

l3[[All, 3]] /= Lookup[lookuptable, l2[[All, 1]]];

the array l3 were still packed (at least if no division by zero occurs). Moreover, l3[[All, 3]] /= [...] would use a more efficient code branch that implements this with compiled C/C++ loops of division on piecewise contiguous memory. At least for floating point data, piecewise contiguous divisions are easy to vectorize for the compiler, so it very likely that the employed library function is vectorized.

So the point was to write it to always perform well - and to perform very well in certain common situations, e.g., involving packed arrays.

$\endgroup$
4
  • $\begingroup$ Thanks Henrik. Gracias Tocayo! worked fine. $\endgroup$ Jun 7, 2018 at 18:21
  • $\begingroup$ I'm always glad to be of help! You're welcome! $\endgroup$ Jun 7, 2018 at 19:14
  • 3
    $\begingroup$ For brevity, likely at the cost of some performance, you can eliminate AssociationThread and use simply Lookup[Rule @@@ l1, l2[[All, 1]]] $\endgroup$
    – Mr.Wizard
    Jun 8, 2018 at 6:44
  • 1
    $\begingroup$ I never sacrifice performance for brevity (if I can help it). Well, almost never. ;) $\endgroup$ Jan 22 at 12:16
5
$\begingroup$
Cases[GatherBy[Join[l1, l2], First], {x_, y_} :> {x[[1]], y[[2]], y[[3]]/x[[2]]}]

{{1990, 1, 100/11}, {2000, 2, 200/21}, {2010, 3, 300/31}}

$\endgroup$
4
$\begingroup$
a = Transpose[{Range[1980, 2010, 1], Range[31]}];

b = {{1990, 1, 100}, {2000, 2, 200}, {2010, 3, 300}}

MapThread[{Splice @ Most[#1], Last[#1 / #2]&, {b, Lookup[Rule @@@ a, First /@ b]}]

{{1990, 1, 100/11}, {2000, 2, 200/21}, {2010, 3, 300/31}}

$\endgroup$
4
$\begingroup$
l1 = Transpose[{Range[1980, 2010, 1], Range[31]}]
l2 = {{1990, 1, 100}, {2000, 2, 200}, {2010, 3, 300}}

Using Lookup:

lut = Rule @@@ l1;
#/{1, 1, Lookup[lut, First@#, 1]} & /@ l2

Using PositionIndex:

pi = PositionIndex[First /@ l1];
#/{1, 1, First@pi[First@#]} & /@ l2

Result:

{{1990, 1, 100/11}, {2000, 2, 200/21}, {2010, 3, 300/31}}

$\endgroup$
3
$\begingroup$
l1 = Transpose[{Range[1980, 2010, 1], Range[31]}];
l2 = {{1990, 1, 100}, {2000, 2, 200}, {2010, 3, 300}};

Using Cases and ReplacePart:

ll = Cases[l1, s_ /; ! FreeQ[s, Alternatives @@ l2[[All, 1]]]];

ReplacePart[l2, {i_, 3} :> l2[[i, 3]]/ll[[i, 2]]]

{{1990, 1, 100/11}, {2000, 2, 200/21}, {2010, 3, 300/31}}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.