1
$\begingroup$

I have a list of values and am interested in finding the contiguous region above a certain threshold.

I have gotten this far,but cannot seem to find the correct way to find the edges. Note: It is not a given that all values above threshold will be contiguous, but it seemed to be promising.


yvals = {7.4336*10^-8,8.42074*10^-8,9.533*10^-8,1.07854*10^-7,1.21948*10^-7,1.37797*10^-7,1.55609*10^-7,1.75613*10^-7,1.98065*10^-7,2.23248*10^-7,2.51475*10^-7,2.83095*10^-7,3.18491*10^-7,3.58089*10^-7,4.02359*10^-7,4.51819*10^-7,5.07043*10^-7,5.6866*10^-7,6.37367*10^-7,7.13928*10^-7,7.99187*10^-7,8.94069*10^-7,9.9959*10^-7,1.11687*10^-6,1.24712*10^-6,1.3917*10^-6,1.55207*10^-6,1.72984*10^-6,1.92676*10^-6,2.14476*10^-6,2.38593*10^-6,2.65257*10^-6,2.94715*10^-6,3.27241*10^-6,3.6313*10^-6,4.02702*10^-6,4.46308*10^-6,4.94327*10^-6,5.4717*10^-6,6.05284*10^-6,6.69151*10^-6,7.39295*10^-6,8.16282*10^-6,9.00723*10^-6,9.93277*10^-6,0.0000109466,0.0000120563,0.0000132703,0.0000145973,0.0000160471,0.0000176298,0.0000193565,0.000021239,0.0000232901,0.0000255232,0.0000279531,0.0000305951,0.0000334659,0.0000365832,0.0000399659,0.0000436341,0.0000476093,0.0000519141,0.0000565727,0.000061611,0.0000670559,0.0000729365,0.0000792833,0.0000861284,0.0000935062,0.000101452,0.000110005,0.000119204,0.000129092,0.000139713,0.000151113,0.000163341,0.000176448,0.000190488,0.000205517,0.000221592,0.000238776,0.000257132,0.000276726,0.000297627,0.000319906,0.000343638,0.000368901,0.000395773,0.000424337,0.000454678,0.000486885,0.000521047,0.000557257,0.000595612,0.000636209,0.000679148,0.000724533,0.000772467,0.000823058,0.000876415,0.000932647,0.000991868,0.00105419,0.00111973,0.0011886,0.00126091,0.00133679,0.00141635,0.00149971,0.00158698,0.00167828,0.00177373,0.00187343,0.0019775,0.00208605,0.00219918,0.002317,0.0024396,0.00256709,0.00269955,0.00283707,0.00297974,0.00312762,0.00328079,0.00343931,0.00360324,0.00377263,0.00394751,0.00412791,0.00431387,0.00450538,0.00470245,0.00490508,0.00511325,0.00532691,0.00554604,0.00577058,0.00600045,0.00623558,0.00647588,0.00672124,0.00697153,0.00722662,0.00748637,0.00775061,0.00801917,0.00829184,0.00856843,0.00884871,0.00913245,0.00941941,0.0097093,0.0100019,0.0102968,0.0105938,0.0108926,0.0111928,0.0114941,0.0117961,0.0120985,0.0124009,0.012703,0.0130042,0.0133043,0.0136027,0.0138992,0.0141933,0.0144846,0.0147726,0.0150569,0.0153371,0.0156127,0.0158834,0.0161486,0.016408,0.0166612,0.0169077,0.0171472,0.0173792,0.0176033,0.0178191,0.0180263,0.0182246,0.0184135,0.0185928,0.018762,0.018921,0.0190694,0.0192069,0.0193334,0.0194485,0.0195521,0.019644,0.019724,0.0197919,0.0198476,0.0198911,0.0199222,0.0199409,0.0199471,0.0199409,0.0199222,0.0198911,0.0198476,0.0197919,0.019724,0.019644,0.0195521,0.0194485,0.0193334,0.0192069,0.0190694,0.018921,0.018762,0.0185928,0.0184135,0.0182246,0.0180263,0.0178191,0.0176033,0.0173792,0.0171472,0.0169077,0.0166612,0.016408,0.0161486,0.0158834,0.0156127,0.0153371,0.0150569,0.0147726,0.0144846,0.0141933,0.0138992,0.0136027,0.0133043,0.0130042,0.012703,0.0124009,0.0120985,0.0117961,0.0114941,0.0111928,0.0108926,0.0105938,0.0102968,0.0100019,0.0097093,0.00941941,0.00913245,0.00884871,0.00856843,0.00829184,0.00801917,0.00775061,0.00748637,0.00722662,0.00697153,0.00672124,0.00647588,0.00623558,0.00600045,0.00577058,0.00554604,0.00532691,0.00511325,0.00490508,0.00470245,0.00450538,0.00431387,0.00412791,0.00394751,0.00377263,0.00360324,0.00343931,0.00328079,0.00312762,0.00297974,0.00283707,0.00269955,0.00256709,0.0024396,0.002317,0.00219918,0.00208605,0.0019775,0.00187343,0.00177373,0.00167828,0.00158698,0.00149971,0.00141635,0.00133679,0.00126091,0.0011886,0.00111973,0.00105419,0.000991868,0.000932647,0.000876415,0.000823058,0.000772467,0.000724533,0.000679148,0.000636209,0.000595612,0.000557257,0.000521047,0.000486885,0.000454678,0.000424337,0.000395773,0.000368901,0.000343638,0.000319906,0.000297627,0.000276726,0.000257132,0.000238776,0.000221592,0.000205517,0.000190488,0.000176448,0.000163341,0.000151113,0.000139713,0.000129092,0.000119204,0.000110005,0.000101452,0.0000935062,0.0000861284,0.0000792833,0.0000729365,0.0000670559,0.000061611,0.0000565727,0.0000519141,0.0000476093,0.0000436341,0.0000399659,0.0000365832,0.0000334659,0.0000305951,0.0000279531,0.0000255232,0.0000232901,0.000021239,0.0000193565,0.0000176298,0.0000160471,0.0000145973,0.0000132703,0.0000120563,0.0000109466,9.93277*10^-6,9.00723*10^-6,8.16282*10^-6,7.39295*10^-6,6.69151*10^-6,6.05284*10^-6,5.4717*10^-6,4.94327*10^-6,4.46308*10^-6,4.02702*10^-6,3.6313*10^-6,3.27241*10^-6,2.94715*10^-6,2.65257*10^-6,2.38593*10^-6,2.14476*10^-6,1.92676*10^-6,1.72984*10^-6,1.55207*10^-6,1.3917*10^-6,1.24712*10^-6,1.11687*10^-6,9.9959*10^-7,8.94069*10^-7,7.99187*10^-7,7.13928*10^-7,6.37367*10^-7,5.6866*10^-7,5.07043*10^-7,4.51819*10^-7,4.02359*10^-7,3.58089*10^-7,3.18491*10^-7,2.83095*10^-7,2.51475*10^-7,2.23248*10^-7,1.98065*10^-7,1.75613*10^-7,1.55609*10^-7,1.37797*10^-7,1.21948*10^-7,1.07854*10^-7,9.533*10^-8,8.42074*10^-8,7.4336*10^-8};

PositionOfMax = 201;

Threshold = 0.00630783;

ValuesAboveThreshold = {False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,True,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False,False}
```
Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
2
$\begingroup$

Plotting yvals, we can see that the function is strictly decreasing when you move away from the maximum, so the region in this case is continuous: ListLinePlot[{yvals, ConstantArray[threshold, Length@yvals]}]

enter image description here

and we can just find the first and last index at or above the threshold:

indicesAboveThresh = Flatten[Position[yvals, n_ /; n >= threshold]];
{First[indicesAboveThresh], Last[indicesAboveThresh]}

If the region was not continuous, we would have to do sort of what you were saying. Start a search going left and right from the maximum until we are no longer at or above the threshold. Here is a really ugly implementation of that search that returns the furthest left and right indices that are above or at the threshold, starting from the position of the maximum value in the list. Someone can probably come along and make a much more elegant implementation:

FindRegionAboveThreshold[list_, thr_] := Module[{l = list},
  maxLoc = Ordering[l][[-1]];
  maxVal = l[[maxLoc]];
  
  val = maxVal;
  indexRight = maxLoc;
  While[val >= thr,
   indexRight++;
   val = l[[indexRight]];
   ];
  indexRight--;
  
  val = maxVal;
  indexLeft = maxLoc;
  While[val >= thr,
   indexLeft--;
   val = l[[indexLeft]];
   ];
  indexLeft++;
  {indexLeft, indexRight}
  ]

Edit: I believe this should also return the largest continuous region around the maximum that is above or at the threshold (but I haven't really been able to test it very much):

maxLoc= Last@Ordering[yvals];
Select[MinMax[#] & /@ Split[Flatten[Position[# >= threshold & /@ yvals, True]], #2 - #1 < 2 &], #[[1]] <= maxLoc <= #[[2]] &]

Second edit: A cleaner way to do this is to split the list into 2 parts, the part left of the peak and the part right of the peak. Reverse the part left of the peak so that it is ordered by how far it is from the peak, and test if each value is greater than threshold. The position of the first 'False' in 'rhs' and 'lhs' corresponds to the distance from the peak to the first value below threshold, so the last value above threshold on each side is just the first False position minus 1:

rhs = # >= threshold & /@ yvals[[maxLoc + 1 ;; All]];
lhs = # >= threshold & /@ Reverse@yvals[[1 ;; maxLoc - 1]];
Flatten[maxLoc + {-(FirstPosition[lhs, False] - 1), FirstPosition[rhs, False] - 1}]
Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Also note I use threshold with a lowercase t, because Threshold with a capital T is a built in function and so shouldn't be used as a variable name $\endgroup$
    – ydd
    Jun 1 at 17:24
  • $\begingroup$ Your "ugly" implementation works :) Thank you! $\endgroup$
    – Indiana
    Jun 1 at 18:02
1
$\begingroup$

Making a lot of use of this function today:

(*https://mathematica.stackexchange.com/a/10653*)
(*Zero crossings in a list*)
davidZC2[l_] := 
 SparseArray[#]["AdjacencyLists"] & /. 
  SApos_ :> 
   With[{c = SApos[l]}, {c[[#]], c[[# + 1]]}\[Transpose] &@
     SApos@Differences@Sign@l[[c]]]

The following returns the indices of the edges of the regions above/below the threshold:

davidZC2[yvals - 0.00630783]

(*  {{140, 141}, {261, 262}}  *)

yvals[[1]] - 0.00630783 evaluates to a negative number showing that the list starts below the threshold, goes above from index 141 to 261, then goes below again. So only one run above the threshold.

The following returns the runs of the elements that are >= the threshold:

Merge[UnitStep[First@# - 0.00630783] -> # & /@ 
   SplitBy[yvals, UnitStep[# - 0.00630783] &], List][1]
(*  <output omitted>  *)

Use Sign instead of UnitStep if you want only elements that are strictly > the threshold. (There's no difference in the case of the OP's data yvals.)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.