4
$\begingroup$

Suppose that I have the following lists of entries,

X1={1,2,3,4,5,6,7,8,9,10};
X2={2,4,6,1,10,12,3,16,18,20};
X3={4,8,12,5,20,24,6,24,20,30};
X4={5,10,16,3,30,30,6,25,25,40};

The entries of X1 represent the values of a physical phenomenon of different points at time t=1. X2 represents the values of the same values at time t=2 and the same for t=3 and t=4.

Now, I would like to find the position of the values that are fluctuating from time t=1 to time t=4.

Of course, in practice my data are huge. These data are just arbitrary values.

$\endgroup$
2
  • $\begingroup$ by fluctuating you mean? $\endgroup$
    – kglr
    Jun 15, 2018 at 7:14
  • $\begingroup$ I mean not increasing or decreasing constantly. Shown variational behaviour $\endgroup$
    – KratosMath
    Jun 15, 2018 at 7:22

4 Answers 4

4
$\begingroup$
fluctuatingPos1 = Pick[Range @ Length @ #, 
   MatchQ[#, {___, a_, ___, b_, ___, c_, ___} /; a < b > c || a > b < c]& /@ #] & @
    Transpose[#]&; 
fluctuatingPos2 =  Flatten@Position[#, {___, a_, ___, b_, ___, c_, ___} /; 
        a < b > c || a > b < c, 1] & @ Transpose[#] &;
fluctuatingPos3 = Flatten @ Position[Unitize[Length[DeleteDuplicates[#]] -  1 & /@ 
  (Sign[Differences[#]] & /@ Transpose[#])], 1, 1] &;
fluctuatingPos4 = Flatten @ Position[(Boole @ Nor[OrderedQ[#], OrderedQ[Reverse@#]] &) /@ 
 Transpose[#], 1] &;

fluctuatingPos1[{X1, X2, X3, X4}]

{4, 7}

Equal[fluctuatingPos1@#, fluctuatingPos2@#, fluctuatingPos3@#, 
  fluctuatingPos4@#] &@{X1, X2, X3, X4}

True

SeedRandom[1]
Y = RandomInteger[10, {1000, 1000}];
{r1 = fluctuatingPos1[Y]; // AbsoluteTiming // First,
 r2 = fluctuatingPos2[Y]; // AbsoluteTiming // First,
 r3 = fluctuatingPos3[Y]; // AbsoluteTiming // First,
 r4 = fluctuatingPos4[Y]; // AbsoluteTiming // First}

{0.747986, 0.755511, 0.031081, 0.009527}

r1 == r2 == r3 == r4

True

$\endgroup$
3
  • $\begingroup$ Thanks. It works for any lists.But could you explain the difference between these functions? $\endgroup$
    – KratosMath
    Jun 15, 2018 at 7:43
  • $\begingroup$ @MsenRezaee, I added timings for a large input list. $\endgroup$
    – kglr
    Jun 15, 2018 at 8:20
  • $\begingroup$ Great! It worked fine for me. $\endgroup$
    – KratosMath
    Jun 15, 2018 at 8:31
2
$\begingroup$

It sound like you want to fit a linear growth model and pick out those data sets that have high error variance. This can be done with LinearModelFit.

Here, I merge you data into a single matrix and add also a list tlist for the time points of your measurements.

X = Transpose[{X1, X2, X3, X4}];
tlist = {1, 2, 3, 4};

This computes fitted models for you data.

fit = LinearModelFit[Transpose[{tlist, #}], {t, 1}, t] &;
data = fit /@ X;

And this returns the estimated error variances. High values mean high fluctuation:

variances = #["EstimatedVariance"] & /@ data

{0.1, 0.4, 0.75, 4.35, 3.75, 3.6, 4.5, 7.35, 4.5, 3.15544*10^-29}

So given a threshold, you can find the position of the data sets with higher variance as threshold with

threshold = 0.5
Flatten[Position[variances, _?(# > threshold &)]]
$\endgroup$
2
  • $\begingroup$ Very nice and professional! $\endgroup$
    – KratosMath
    Jun 15, 2018 at 13:45
  • $\begingroup$ Huh... professional? Thanks for the flowers! $\endgroup$ Jun 15, 2018 at 13:55
2
$\begingroup$

You can use a high-pass filter. It lets quick variations in the data pass through. This is what the four datasets look like when high-pass filtered:

ListLinePlot[HighpassFilter[#, 2] & /@ {X1, X2, X3, X4}]

High-pass filtered datasets

I have used a cutoff frequency of 2 here, you may want to adjust that according to your data.

To get some quantification, you could sum over the absolute values, for example:

N@Total[Abs@HighpassFilter[#, 2]] & /@ {X1, X2, X3, X4}
(* {0.477118, 19.1493, 37.1699, 49.6903} *)
$\endgroup$
2
$\begingroup$

I seem to be a bit late to this thread, but if you're just looking to test whether your list is monotonically increasing or decreasing, I think there's a much simpler way:

isMonotonic := Length@DeleteCases[
 DeleteDuplicates@Sign@Differences@#
 , 0] < 2 &

isMonotonic /@ {X1, X2, X3, X4}
{True, False, False, False}

Here, we're just using Sign to take the sign of the Differences of the given list, and checking whether both 1 and -1 appear.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.