8
$\begingroup$

Consider :

list = {100,102,103,99,106,107,104,112}

I want to consider, extract only the "big moves" of that time series. In this case the moves bigger then 5 from the moving point of reference.

The pseudo code would go as such :

Let's assume you have the data x(t) for t=1 to T,

You start from the first data point, z(1) = x(1)

if abs(x(2)-z(1))> delta, 
    z(2)= x(2), 
elseif abs(x(3)-z(1))> delta, 
    z(2)= x(3), 
elseif abs(x(4)-z(1))> delta, 
    z(2)= x(4),    
    if abs(x(5)-z(2))> delta, 
        z(3)= x(5), 
    elseif abs(x(6)-z(2))>delta, 
        z(3)= x(6),
        if abs(x(7)-z(3))>delta, 
            z(4)=x(7)
        endif
    endif
endif

I tried to write it with an If statement, but I don't know when my test is going to pass thus, how many Ifs to nest.

If[Abs[list[[2]]-list[[1]]]>5,
   list[[2]],
             If[Abs[list[[3]]-list[[1]]>5,
             list[[3]]....

Here this process will stop at list[[5]] with Abs[list[[5]]-list[[1]]=6

At this point, list[[5]] becomes my new point of reference :

 If[Abs[list[[6]]-list[[5]]]>5,
     list[[6]],
              If[Abs[list[[7]]-list[[5]]>5,
             list[[7]]....

So my desired output is the following :

{100,106,112}

I am pretty sure, this type of operation has a name but I don't know it. It felt like a While[] or For[] but I can`t figure it out. Please let me know how to improve the question.

$\endgroup$
5
  • $\begingroup$ You meant "You start from the first data point, z(1) = x(1)"? $\endgroup$
    – Rojo
    Commented Feb 25, 2012 at 0:44
  • $\begingroup$ @Rojo, Yes ! Thank you, Edited. $\endgroup$
    – 500
    Commented Feb 25, 2012 at 0:45
  • $\begingroup$ 500, apparently I misunderstood your question, so please disregard my answers for the time being. I'll leave them for reference. I am too tired to fix it now. $\endgroup$
    – Mr.Wizard
    Commented Feb 25, 2012 at 3:14
  • 1
    $\begingroup$ 500, just FYI, you're getting you've been using backquotes () instead of an apostrophe (') in contractions. For example "dont" instead of "don't", which causes problems with the formatting of your posts. $\endgroup$ Commented Feb 25, 2012 at 5:36
  • $\begingroup$ This reminds me of things like KagiChart and RenkoChart which have similar behavior. It's not easy to clearly and concisely describe them either. $\endgroup$ Commented Feb 25, 2012 at 5:38

6 Answers 6

10
$\begingroup$

In contrast to the very fast methods presented, let me give you a maybe more easy to understand solution (while this in the eye of the beholder). Basically it's sort of the way I would implement it in Haskell.

Assume you want to solve your problem step by step and in each step you have your data in the form {res, in} where res is your current solution and in is the rest of the input which needs to be processed. The initial situation is that the list res contains your first element and in contains the rest of the input. Let's go through the two possible situations:

  • Since res is your current output, you always want to compare the last element of this with the first element of your in list. When the absolute difference is larger then 5, you want to append this first element of in to your res list.

  • In all other cases, you just want to throw away the first element of in and leave res untouched.

You want to iterate this as long as there are elements in your input. Thats all, and you can write it directly down:

step[{res_, in_}] := {Append[res, First[in]], Rest[in]} /; 
  Abs[Last[res] - First[in]] > 5;

step[{res_, in_}] := {res, Rest[in]};

NestWhile[step, {{100}, {102, 103, 99, 106, 107, 104, 112}}, 
 Length[Last[#]] > 0 &]

(* {{100, 106, 112}, {}} *)

You see that after your input list is empty, you have collected your result.

$\endgroup$
8
$\begingroup$

Something along these lines could work:

Reap[
  Sow[ z = list[[1]] ];
  Scan[If[Abs[# - z] > delta, Sow[z = #]] &, list]
][[2, 1]]
$\endgroup$
0
7
$\begingroup$

I favor Fold over Scan. I find it faster, and it does without an additional variable.

Reap[
  Fold[If[Abs[#2 - #] > delta, Sow[#2], #] &, Infinity, list];
][[2, 1]]

500 asked:

How could I now store as a sublist every element skipped in between the z points in addition to the the z point themselves. I would like to have the list of the coordinates that do not pass the test, but sublists for each group of point skipped in between two Sow[].

This is the cleanest method I could think of:

list = {100, 102, 103, 99, 106, 107, 104, 107, 112};

idx =
  Reap[i = 0;
    Fold[If[++i; Abs[#2 - #] > delta, Sow[i]; #2, #] &, Infinity, list];
  ][[2, 1]];

MapThread[list[[# ;; #2]] &, {idx, Append[Rest@idx - 1, -1]}]
{{100, 102, 103, 99}, {106, 107, 104, 107}, {112}}

The output is a list of sublists, each starting with a "z point" and continuing with intervening elements before the next point.

$\endgroup$
12
  • $\begingroup$ @500 I just added a much faster, if admittedly more specific answer. $\endgroup$
    – Mr.Wizard
    Commented Feb 25, 2012 at 1:25
  • $\begingroup$ This doesn't give the answer I expect for {100, 102, 103, 99, 106, 107, 104, 107, 112}. $\endgroup$ Commented Feb 25, 2012 at 1:45
  • $\begingroup$ @Brett Since two notes of incorrect output have been made I perhaps didn't understand the question. What output do you expect for that list? $\endgroup$
    – Mr.Wizard
    Commented Feb 25, 2012 at 2:51
  • $\begingroup$ Btw, I had written a similar solution and deleted it in a panick attack. It had a bug however not sowing the first element, that you neatly dealt with with that Infinity. +1 for the Infinity, hehe $\endgroup$
    – Rojo
    Commented Feb 25, 2012 at 4:59
  • $\begingroup$ I expect {100, 106, 112}. See the responses from @halirutan and @Rojo on your other answer. $\endgroup$ Commented Feb 25, 2012 at 5:33
6
$\begingroup$

Here is a slightly faster version based on FoldList and Split.

Split[
  FoldList[If[Abs[#2 - #] > delta, #2, #] &, Infinity, list]
][[2 ;;, 1]]
$\endgroup$
5
  • $\begingroup$ I think your first function works not correctly. Can you test the input {20, 56, 57, 62, 20}? $\endgroup$
    – halirutan
    Commented Feb 25, 2012 at 1:58
  • $\begingroup$ @halirutan Since two notes of incorrect output have been made I perhaps didn't understand the question. What output do you expect for that list? $\endgroup$
    – Mr.Wizard
    Commented Feb 25, 2012 at 2:51
  • 2
    $\begingroup$ @MrWizard I expect {20, 56, 62, 20}. You misunderstood maybe, that after the test for (56,57) fails, 56 is kept as *reference element` until a test succeeds. $\endgroup$
    – halirutan
    Commented Feb 25, 2012 at 3:08
  • 1
    $\begingroup$ @Mr.Wizard, this one yes shows you misunderstood. It doesn't do the same as your previous answer. This one only collects those contiguous jumps above delta, but he wants to go dropping values until you get to the delta difference. So if delta=3, {0, 1, 2, 3, 4, 5, 6,7,8} should give {0, 4, 8} if I understood $\endgroup$
    – Rojo
    Commented Feb 25, 2012 at 5:02
  • $\begingroup$ @Mr.Wizard yeah, they seem to work well if I understood the problem. I was wrong with my comment on the other post the other day, I had a blank and thought Sow as something that returned Null. +1 on this one now $\endgroup$
    – Rojo
    Commented Feb 27, 2012 at 0:19
5
$\begingroup$

Here's another approach, based on a recursive function and pattern matching.

MinDifference[list_, d_] := MinDifference[{First[list]}, Rest[list], d]

MinDifference[store_, {}, d_] := store

MinDifference[{store___, s_}, {v_, list___}, d_] /; Abs[v-s] > d := 
   MinDifference[{store, s, v}, {list}, d]

MinDifference[{store___}, {v_, list___}, d_] := 
   MinDifference[{store}, {list}, d]

MinDifference[{100, 102, 103, 99, 106, 107, 104, 112}, 5]
{100, 106, 112}
MinDifference[{100, 102, 103, 99, 106, 107, 104, 107, 112}, 5]
{100, 106, 112}
$\endgroup$
3
$\begingroup$

Another yet-another possibility is ReplaceRepeated:

Define

crossings = ({{First[#1]}, #1}) //. 
({a_, {Shortest[b__], e___}} /; (Abs[First@{b} - Last@{b}] > #2) 
 :> {Insert[a, Last@{b}, -1], {Last@{b}, e}}) &

Usage:

 crossings @@ {list, 5}   

gives

{{100, 106, 112}, {112}}

the first part of which gives the desired result:

 crossings @@ {list, 5} //First
 (* ==> {{100, 106, 112} *) 

An alternative is using ReplaceAll in combination with FixedPointList:

 crossingList[list_, delta_] := 
 First /@ Most@
 FixedPointList[#1 /. ({Shortest[a__], b___} /; 
 (Abs[First@{a} - Last@{a}] > delta) 
 :> {Last@{a}, b}) &, list]
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.