# Speed optimization of Mean calculation in a previous calculated data range

I have following problem: My calculations work quite well with the shown testdata, the only problem is the speed of it. My original dataset is about 64000 datapoints, and it takes several minutes to calculate the following script. I have the clue that my 2 For loops could be the reason, has anyone an idea how I could speed it up??

I have following datasets

Signal={{1., 0.5}, {1.1, 0.721739}, {1.2, 0.860952}, {1.3, 0.932368}, {1.4,
0.966584}, {1.5, 0.982954}, {1.6, 0.990987}, {1.7, 0.995064}, {1.8,
0.997207}, {1.9, 0.998372}, {2., 0.999024}, {2.1, 0.999401}, {2.2,
0.999624}, {2.3, 0.999759}, {2.4, 0.999842}, {2.5, 0.999895}, {2.6,
0.999929}, {2.7, 0.999951}, {2.8, 0.999966}, {2.9, 0.999976}, {3.,
0.999983}, {3.1, 0.999988}, {3.2, 0.999991}, {3.3, 0.999993}, {3.4,
0.999995}, {3.5, 0.999996}, {3.6, 0.999997}, {3.7, 0.999998}, {3.8,
0.999998}, {3.9, 0.999999}, {4., 0.999999}, {4.1, 0.999999}, {4.2,
0.999999}, {4.3, 1.}, {4.4, 1.}, {4.5, 1.}, {4.6, 1.}, {4.7,
1.}, {4.8, 1.}, {4.9, 1.}, {5., 1.}, {5.1, 1.}, {5.2, 1.}, {5.3,
1.}, {5.4, 1.}, {5.5, 1.}, {5.6, 1.}, {5.7, 1.}, {5.8, 1.}, {5.9,
1.}, {6., 1.}, {6.1, 1.}, {6.2, 1.}, {6.3, 1.}, {6.4, 1.}, {6.5,
1.}, {6.6, 1.}, {6.7, 1.}, {6.8, 1.}, {6.9, 1.}, {7., 1.}, {7.1,
1.}, {7.2, 1.}, {7.3, 1.}, {7.4, 1.}, {7.5, 1.}, {7.6, 1.}, {7.7,
1.}, {7.8, 1.}, {7.9, 1.}, {8., 1.}, {8.1, 1.}, {8.2, 1.}, {8.3,
1.}, {8.4, 1.}, {8.5, 1.}, {8.6, 1.}, {8.7, 1.}, {8.8, 1.}, {8.9,
1.}, {9., 1.}, {9.1, 1.}, {9.2, 1.}, {9.3, 1.}, {9.4, 1.}, {9.5,
1.}, {9.6, 1.}, {9.7, 1.}, {9.8, 1.}, {9.9, 1.}, {10., 1.}, {10.1,
1.}, {10.2, 1.}, {10.3, 1.}, {10.4, 1.}, {10.5, 1.}, {10.6,
1.}, {10.7, 1.}, {10.8, 1.}, {10.9, 1.}, {11., 1.}, {11.1,
1.}, {11.2, 1.}, {11.3, 1.}, {11.4, 1.}, {11.5, 1.}, {11.6,
1.}, {11.7, 1.}, {11.8, 1.}, {11.9, 1.}, {12., 1.}, {12.1,
1.}, {12.2, 1.}, {12.3, 1.}, {12.4, 1.}, {12.5, 1.}, {12.6,
1.}, {12.7, 1.}, {12.8, 1.}, {12.9, 1.}, {13., 1.}, {13.1,
1.}, {13.2, 1.}, {13.3, 1.}, {13.4, 1.}, {13.5, 1.}, {13.6,
1.}, {13.7, 1.}, {13.8, 1.}, {13.9, 1.}, {14., 1.}, {14.1,
1.}, {14.2, 1.}, {14.3, 1.}, {14.4, 1.}, {14.5, 1.}, {14.6,
1.}, {14.7, 1.}, {14.8, 1.}, {14.9, 1.}, {15., 1.}, {15.1,
1.}, {15.2, 1.}, {15.3, 1.}, {15.4, 1.}, {15.5, 1.}, {15.6,
1.}, {15.7, 1.}, {15.8, 1.}, {15.9, 1.}, {16., 1.}, {16.1,
1.}, {16.2, 1.}, {16.3, 1.}, {16.4, 1.}, {16.5, 1.}, {16.6,
1.}, {16.7, 1.}, {16.8, 1.}, {16.9, 1.}, {17., 1.}, {17.1,
1.}, {17.2, 1.}, {17.3, 1.}, {17.4, 1.}, {17.5, 1.}, {17.6,
1.}, {17.7, 1.}, {17.8, 1.}, {17.9, 1.}, {18., 1.}, {18.1,
1.}, {18.2, 1.}, {18.3, 1.}, {18.4, 1.}, {18.5, 1.}, {18.6,
1.}, {18.7, 1.}, {18.8, 1.}, {18.9, 1.}, {19., 1.}, {19.1,
1.}, {19.2, 1.}, {19.3, 1.}, {19.4, 1.}, {19.5, 1.}, {19.6,
1.}, {19.7, 1.}, {19.8, 1.}, {19.9, 1.}, {20., 1.}}


MicrowavePulse={{1, 1}, {2, 1}, {3, 1}, {4, 1}, {5, 1}, {6, 0}, {7, 0}, {8, 0}, {9,0}, {10, 0}, {11, 1}, {12, 1}, {13, 1}, {14, 1}, {15, 1}, {16, 0},{17, 0}, {18, 0}, {19, 0}, {20, 0}}


My goal is to get the Meanof Signal data, in the range whenever the MicrowavePulse is on and off. That means I want to get 4x (in this case) the Meanof all Signal data points lying from x=1 to x=5, from x=6 to x=10, from x=11 to x=15 and from x=16 till x=20.

I have written a For loop in order to receive sublists, for each time the MW is on or off, that means I get 4 sublists.

NumberMWData = Length[MicrowavePulse]
MicrowavePulse = MicrowavePulse[[All, 2]];
MWListOnOFF := {};
MWOneList := {};
For[i = 1, i <= NumberMWData, i++,
MWOneList = Append[MWOneList, i];
If[i == NumberMWData || (MicrowavePulse[[i]] > 0 &&
MicrowavePulse[[i + 1]] <= 0) || (MicrowavePulse[[i]] <= 0 &&
MicrowavePulse[[i + 1]] > 0),
MWListOnOFF = Append[MWListOnOFF, MWOneList];
MWOneList = {}
]
]

(* {{1, 2, 3, 4, 5}, {6, 7, 8, 9, 10}, {11, 12, 13, 14, 15}, {16, 17, 18,19, 20}} *)


These lists are applied to cut the Signal data in the same x range like the MW pulse in lists, so that I can calculate the Mean in each of the Signal lists and the result are four mean values for the Signal data.

ResultMean := {};
For[i = 1, i <= Length[MWListOnOFF], i++,
start = MWListOnOFF[[i]][[1]];
stop = MWListOnOFF[[i]][[Length[MWListOnOFF[[i]]]]];
ResultMean = Append[ResultMean,Mean[Select[Signal, start <= #[[1]] <= stop &]]];
]

(*{{3., 0.974232}, {8., 1.}, {13., 1.}, {18., 1.}}*)


indices = SplitBy[MicrowavePulse, Last][[All, All, 1]]
(* {{1, 2, 3, 4, 5}, {6, 7, 8, 9, 10}, {11, 12, 13, 14, 15}, {16, 17, 18, 19, 20}} *)

Mean[Signal[[#]]] & /@ indices
(* {{1.2,0.796329},{1.7,0.992917},{2.2,0.99953},{2.7,0.999943}} *)


or

Extract[Signal, List /@ indices, Mean]
(* {{1.2,0.796329},{1.7,0.992917},{2.2,0.99953},{2.7,0.999943}} *)

• Wow, that is great! The first code SplitBy...is super. But the second and third are not giving the right result, do you have a clue why? Nov 10, 2014 at 17:21
• @Jacccy, that's because Signal and MicrowavePulse have different lengths (191 and 20, respectively), and the functions posted are using the first 20 elements of Signal.
– kglr
Nov 10, 2014 at 17:26
• I have one additional question, if I want to split y-values in a specific range e.g (#2 > 0 && #1 <= 0) || (#1 > 0 && #2 <= 0)] & where do I have to add this in the code? And is my code correct? Nov 12, 2014 at 12:32
• Jacccy, if you want to split yourlist using the splitting rule in your comment, you can do Split[yourlist,(#2 > 0 && #1 <= 0) || (#1 > 0 && #2 <= 0)&]  or Split[yourlist, Times@## <= 0 &].
– kglr
Nov 12, 2014 at 13:32
• @Jacccy, you do need integers as part indices. Not sure how you would like to use 1.1, 2.3 etc as part indices --maybe you can use Round or IntegerPart to turn real values to integers.
– kglr
Nov 12, 2014 at 16:52