5
$\begingroup$
ListPlot[list, Joined -> True, PlotRange -> Full]

enter image description here

I would like a simple method to get the positions within the list where the values deviate a lot from their previous values. If I just use a threshold, I might miss some peaks or introduce wrong results as the values before a peak can vary.

Here is the list:

list = Import["https://pastebin.com/raw/ncJDt0L4", "List"]

I was thinking of using a numeric gradient, but I don't know if Mathematica has a build in function and also if this is really the best method.

EDIT:

I forgot to mention: A peak should always be chosen with respect to the previous points which lie after the last detected peak, or in other terms: All values before the last detected peak should not be considered in evaluating the next peak. Only the values after the last detected peak are of importance.

The first small peaks before the first big peak should not be detexted as peaks since the data fluctuates a lot there

...and the last little peak should not be counted as peaks, because there cannot be two peaks when the distance (x) between the peak position is less than 3.

$\endgroup$
  • $\begingroup$ Look up ListConvolve. $\endgroup$ – David G. Stork Aug 14 '18 at 17:57
  • $\begingroup$ @rhermans Thanks for the edit ! :) $\endgroup$ – james Aug 14 '18 at 18:29
  • $\begingroup$ @DavidG.Stork Interesting, thanks for the note. How would you use it in this case? $\endgroup$ – james Aug 14 '18 at 18:30
  • $\begingroup$ ListConvlve the data with a "center-surround" filter that will "pick out" peaks. $\endgroup$ – David G. Stork Aug 14 '18 at 23:02
5
+100
$\begingroup$

FindPeaks is the perfect solution as it has all the options to fine tune the condition you require for a peak:

list = Import["https://pastebin.com/raw/ncJDt0L4", "List"];

peaks = FindPeaks[list, 3, 12];
ListLogPlot[{list, peaks}, PlotRange -> Full, Joined -> {True, False},
  PlotStyle -> {Thin, Directive[Red, PointSize[Large]]}]

enter image description here

Just play around with the values for Gaussain blurring, minimum sharpness and threshold until you get the performance you want.

Edit

The log-plot is very useful for highlighting the reason, why the smallest peaks are also considered a peak. In case you do not consider it a peak, you can easily eliminate the very small but sharp features by also using the threshold, the fourth argument of FindPeaks.

For a direct comparison with other solutions, here is a linear plot:

ListPlot[{list, peaks}, PlotRange -> Full, Joined -> {True, False}, 
 PlotStyle -> {Thin, Directive[Red, PointSize[Large]]}]

enter image description here

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks a lot ! The Log plot is great, but can you also add a linear plot for direct comparaison with the other answers? $\endgroup$ – james Aug 18 '18 at 2:00
  • $\begingroup$ @james Here you go. $\endgroup$ – Johu Aug 18 '18 at 9:36
  • $\begingroup$ Great !! Thank you ! :)) $\endgroup$ – james Aug 18 '18 at 12:56
5
$\begingroup$

Using FindPeaks is a fine solution. This answer responds to the following OP statements/questions.

I would like a simple method to get the positions within the list where the values deviate a lot from their previous values. If I just use a threshold, I might miss some peaks or introduce wrong results as the values before a peak can vary.

I was thinking of using a numeric gradient, but I don't know if Mathematica has a build in function

Get the (consecutive) differences:

ds = Differences[list];
Through[{Min, Max, Mean, Median, StandardDeviation}[ds]]

(* {-24958.8, 25131.1, 0.526443, -1.1353, 2896.09} *)

Pick peak positions with a threshold (0.06 here):

ds2 = Select[ds, # > 0 &];
pos = Pick[Range[2, Length[list]], 
  Map[ # > Mean[ds2] + 0.06*StandardDeviation[ds2] &, ds]]

Plot data and peaks:

list2 = Transpose[{Range[Length[list]], list}];
ListLinePlot[list2, PlotRange -> Full, 
 Epilog -> {Red, PointSize[Large], Point[list2[[pos]]]}]

enter image description here

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks a lot for your answer. Do you see a method of how to avoid those initial peaks, because they should theoretically not be seen as peaks. $\endgroup$ – james Aug 14 '18 at 20:05
  • $\begingroup$ see Edited question $\endgroup$ – james Aug 14 '18 at 20:13
  • $\begingroup$ @james I thought the initial peaks are important! ;) And yes there is a method to avoid them... $\endgroup$ – Anton Antonov Aug 14 '18 at 20:19
  • $\begingroup$ okay, awesome. Looking forward seeing it. $\endgroup$ – james Aug 14 '18 at 20:25
  • $\begingroup$ which method is it that you were thinking of ? $\endgroup$ – james Aug 15 '18 at 8:55
4
$\begingroup$

I think that you are likely looking for some combination of the parameters of the FindPeak function, for example:

list = Import["https://pastebin.com/raw/ncJDt0L4", "List"];

peaks = FindPeaks[list, Automatic, Automatic, 500];
ListLinePlot[
 list,
 PlotRange -> Full,
 Epilog -> {Red, PointSize[Large], Point[peaks]}
 ]

Mathematica graphics

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks a lot for your answer. Do you see a method of how to avoid those initial peaks, because they should theoretically not be seen as peaks. $\endgroup$ – james Aug 14 '18 at 20:05
  • $\begingroup$ @james Why shouldn't they be seen as peaks? $\endgroup$ – C. E. Aug 14 '18 at 20:06
  • $\begingroup$ The values before the first peak fluctuate and hence should not be seen as peaks, but rather as noise with respect to the first peak. I forgot to mention: The peaks should always be chosen with respect to the previous points which lie after the last peak. $\endgroup$ – james Aug 14 '18 at 20:10
  • $\begingroup$ @james Is there a specific minimum separation between peaks before they are seen as separate peaks, rather than part of some larger peak? $\endgroup$ – C. E. Aug 14 '18 at 20:13
  • $\begingroup$ Hmm, good question. Yes, in principle, the minimum separation would probably be 3. $\endgroup$ – james Aug 14 '18 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.