Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I have a problem that I am working on that requires that I compare several lists of overlapping data points. As I traverse one of the lists, I'd like to only compare those data to the other lists, and not itself. So, how does one efficiently do this, especially if it must be done potentially hundreds of times? Obviously, if I can control the order in which I make the comparisons, any method would work as I can limit the number of times this has to be done. But, I can't necessarily guarantee the order.

Now both Drop and Take could be used for this, but I'm interested to know if they are the fastest methods to do this, or is there something else?

share|improve this question
add comment

5 Answers

up vote 12 down vote accepted

You can use Most[list] to do this if you want to drop last element, Rest[list] to drop the first element, and Delete[list, n] to drop the $n^\mathrm{th}$ element.

I verified that none of these unpack packed arrays, so performance should be good. Drop and Take don't unpack either, so the performance of those should be similar too, at least for packed arrays.

share|improve this answer
    
Ah, forgot about Delete. –  rcollyer Jan 18 '12 at 17:22
    
Rest[] drops the first element, being complementary to First[]. Analogously, Most[] pairs nicely with Last[]... –  J. M. Jan 18 '12 at 17:25
    
On Delete[] vs. Drop[]: Delete[Range[5], 3] and Drop[Range[5], {3}] do the same thing, of course. –  J. M. Jan 18 '12 at 17:28
add comment

All the above are useful when you know the position of elements. Another frequent case is when you do not know the position, but know the type or value:

In[1]:= DeleteCases[Range[5], 2]    
Out[1]= {1, 3, 4, 5}

In[2]:= DeleteCases[Range[15], x_ /; 10 < x < 14]    
Out[2]= {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 14, 15}

In[3]:= DeleteCases[{3, rr, "space", 3, 5, t}, "space"]
Out[3]= {3, rr, 3, 5, t}

In[4]:= DeleteCases[{W, XYZ, 3 + I 5, "wer", 1.5, 1/6}, _Complex]
Out[4]= {W, XYZ, "wer", 1.5, 1/6}

In[5]:= DeleteCases[{1, f[2, 3], 4}, f, {2}, Heads -> True]
Out[5]= {1, 2, 3, 4}
share|improve this answer
1  
For numerical lists, the problem is that DeleteCases will unpack. So it is only efficient for unpacked lists and general symbolic expressions. If you deal with numerical lists, there are ways to efficiently extract a position of an element without unpacking (the most efficient being probably the compiled-to-C code). Then use Delete. –  Leonid Shifrin Jan 20 '12 at 21:19
add comment

Using Drop, Take, Most, etc. are fine but also consider the expresiveness of Part or [[]]

Here are some examples:

list = {3, 6, 9, 12};

In[532]:= list[[{1, 2, 3}]]

Out[532]= {3, 6, 9}

In[534]:= list[[1 ;; 3]]

Out[534]= {3, 6, 9}

In[535]:= list[[Range[Length[list] - 1]]]

Out[535]= {3, 6, 9}

In[536]:= list[[{3, 2, 1}]]

Out[536]= {9, 6, 3}

In[537]:= list[[{1, 3, 1, 1, 1}]]

Out[537]= {3, 9, 3, 3, 3}
share|improve this answer
1  
One particularly neat thing about Part[] is that it allows for a parallel assignment construction like list[[1 ;; 3]] = list[[3 ;; 1 ;; -1]]. –  J. M. Jan 26 '12 at 6:15
    
But, how do you return everything but the 9 out of your list using Part? –  rcollyer Jan 26 '12 at 11:03
add comment

An alternative I had in mind when I wrote the question was to use Pick alongside Permutations. Pick allows you to specify a mask that can be used to select elements from a list,

In[] := Pick[(data = {a, b, c, d, e, f}), {1,1,0,0,0,1}, 1]
Out[]:= {a, b, f}

and Permuations can be used to generate such a mask,

In[] := Permutations[{0}~Join~Array[1&, Length@data - 1]]
Out[]:= {{0,1,1,1,1}, {1,0,1,1,1}, {1,1,0,1,1}, {1,1,1,0,1}, {1,1,1,1,0}}

Then,

Pick[
   (data = {a, b, c, d, e, f}), #, 1
 ]& /@ Permutations[{0}~Join~Array[1&, Length@data - 1]]

returns

{{b, c, d, e, f}, {a, c, d, e, f}, 
 {a, b, d, e, f}, {a, b, c, e, f}, 
 {a, b, c, d, f}, {a, b, c, d, e}}

Alternatively, IdentityMatrix[Length@data] could be used

Pick[data, #, 0]& /@ IdentityMatrix[Length@data]

where the pattern has been changed from 1 to 0.

share|improve this answer
    
is it basically just the diagonal that you want excluded @rcollyer? –  Mike Honeychurch Jan 18 '12 at 22:05
    
@MikeHoneychurch, pretty much. But, I expected to have to perform it a lot more often than what I've shown. In review, probably not as much as I thought. –  rcollyer Jan 18 '12 at 22:36
    
rcollyer's message at 7:18 on the 18th appears to be missing a 0 in the mask: Pick[(data = {a, b, c, d, e, f}), {1,1,0,0,1}, 1] produces the message "Pick::incomp: "Expressions {a,b,c,d,e,f} and {1,1,0,0,1} have incompatible shapes"" –  Gary Palmer Jan 25 '12 at 22:54
add comment

It is unclear whether your answer to your question represents the real problem or just an example but another way to get permutations that exclude the diagonal is to use ListCorrelate:

ListCorrelate[{0, 1, 1, 1, 1, 1}, {a, b, c, d, e, f}, 1]

{b + c + d + e + f, a + c + d + e + f, a + b + d + e + f, 
 a + b + c + e + f, a + b + c + d + f, a + b + c + d + e}

That isn't what you want obviously but my long winded explanation starts with the general form of the above:

ListCorrelate[{0, 1, 1, 1, 1, 1}, {a, b, c, d, e, f}, {1, 1}, {a, b, 
  c, d, e, f}, Times, Plus]

{b + c + d + e + f, a + c + d + e + f, a + b + d + e + f, 
 a + b + c + e + f, a + b + c + d + f, a + b + c + d + e}

So you can get an answer by using a function other than Plus in the generalized ListCorrelate:

g[x__] := DeleteCases[{x}, 0] (* or g= DeleteCases[{##}, 0] & *)

ListCorrelate[{0, 1, 1, 1, 1, 1}, {a, b, c, d, e, f}, {1, 1}, {a, b, 
  c, d, e, f}, Times, g]

{{b, c, d, e, f}, {c, d, e, f, a}, {d, e, f, a, b}, {e, f, a, b, 
  c}, {f, a, b, c, d}, {a, b, c, d, e}}

I really like ListCorrelate but unfortunately the generalized form is much slower than the default form -- you may find it up to 2 or more orders of magnitude slower depending on your problem [plea to Wolfram: make the generalized form of ListCorrelate run fast -- there is so much it could be useful for]. So the alternative in a case like this may be to revert to the default form and replace Plus in the result:

ListCorrelate[{0, 1, 1, 1, 1, 1}, {a, b, c, d, e, f}, 1] /. Plus -> List

{{b, c, d, e, f}, {a, c, d, e, f}, {a, b, d, e, f}, {a, b, c, e, 
  f}, {a, b, c, d, f}, {a, b, c, d, e}}
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.