4
$\begingroup$

I have a list dimensions {4, 2, ?, 3}. The ? can be any value from 1 to 5. Here's a simplified example:

data = {{{{-1,0.2,0}},{{-1,0.5,0}}},
{{{1,0.47,0},{-1,0.02,0},{1,1.0,0}},{{-1,0.12,0},{-1,0.11,0},{1,0.,0}}},
{{{1,0.1,0},{1,0.41,0},{-1,0.32,0}},{{1,0.02,0},{1,0.33,0},{1,0.01,0}}},
{{{-1,0.1,0},{-1,0.1,0}},{{1,0.41,0},{-1,0.6,0}}}};

I need to calculate an average progressively on the second value of the last dimension, which I've calculated this way so far:

second = Flatten[data[[All,All,All,2]]];
mean = Accumulate[second]/Range[Length[second]];

Now, I wish to replace the old values in data with this calculated average :

result = {{{{-1,0.2,0}},{{-1,0.35,0}}},
{{{1,0.39,0},{-1,0.2975,0},{1,0.438,0}},{{-1,0.385,0},{-1,0.345714,0},{1,0.3025,0}}},
{{{1,0.28,0},{1,0.293,0},{-1,0.295455,0}},{{1,0.2725,0},{1,0.276923,0},{1,0.257857,0}}},
{{{-1,0.247333,0},{-1,0.238125,0}},{{1,0.248235,0},{-1,0.267778,0}}}};

If the dimensions were known, it wouldn't be a problem. But they aren't. Also, my real dimensions are along the lines of {40, 10000, ?, 3}. I will be running this transformation thousands of time, so speed is key.

EDIT : This is what I've tried and it works, but I'm certain there is a better way

count = 0;
data[[All, All, All, 2]] = Table[++count;
mean[[count]], {n, 4}, {t, 2}, {l, Length[data[[n, t]]]}]
$\endgroup$
5
$\begingroup$

Here is a semi-imperative way to insert the mean values back into data:

Module[{i = 0}, MapAt[mean[[++i]] &, data, {All, All, All, 2}]]

Alternatively, we could perform the complete moving average operation semi-imperatively:

Module[{total = 0, count = 0}
, MapAt[(total += #)/(count += 1)&, data, {All, All, All, 2}]
]
$\endgroup$
  • $\begingroup$ I really like the moving average integrated in, thanks! $\endgroup$ – Lokdal Apr 21 '17 at 16:43
  • $\begingroup$ @Lokdal You are welcome. It is common practice to wait a little while (say, a day or so) before accepting an answer to encourage others to submit even better answers. $\endgroup$ – WReach Apr 21 '17 at 16:52
  • $\begingroup$ I will do that! $\endgroup$ – Lokdal Apr 21 '17 at 17:29
5
$\begingroup$
data[[All, All, All, 2]] = placeholder
ReplacePart[data, Thread[Position[data, placeholder] -> mean]]
$\endgroup$
  • $\begingroup$ Works great! But I'm selecting WReach's answer because it is faster and more elequent. $\endgroup$ – Lokdal Apr 21 '17 at 16:43
  • $\begingroup$ @Lokdal Agreed--it's a great solution! $\endgroup$ – Chris Apr 21 '17 at 22:07
1
$\begingroup$
data[[;;, ;;, ;;, 2]] = Module[{i = 1}, Map[mean[[i++]] &, data[[;;, ;;, ;;, 2]], {-1}]]; 

data

{{{{-1, 0.2, 0}}, {{-1, 0.35, 0}}},   
 {{{1, 0.39, 0}, {-1, 0.2975,  0}, {1, 0.438, 0}},   
  {{-1, 0.385, 0}, {-1, 0.345714, 0}, {1, 0.3025, 0}}},   
 {{{1, 0.28, 0}, {1, 0.293, 0}, {-1, 0.295455,  0}},  
  {{1, 0.2725, 0}, {1, 0.276923, 0}, {1, 0.257857, 0}}},   
 {{{-1, 0.247333, 0}, {-1, 0.238125, 0}},   
  {{1, 0.248235, 0}, {-1, 0.267778, 0}}}}
$\endgroup$
0
$\begingroup$

From your question I do not know if you absolutely need to preserve the structure of your data. If not then consider using this:

Transpose[{#[[All, 1]], mean, #[[All, 3]]}] &@Level[data, {-2}]

(* {{-1, 0.2, 0}, {-1, 0.35, 0}, {1, 0.39, 0}, {-1, 0.2975, 0}, {1, 0.438, 0},
{-1, 0.385, 0}, {-1, 0.345714, 0}, {1, 0.3025, 0}, {1,0.28, 0}, {1, 0.293, 0},
{-1, 0.295455, 0}, {1, 0.2725, 0}, {1,0.276923, 0}, {1, 0.257857, 0}, {-1, 0.247333, 0},
{-1, 0.238125, 0}, {1, 0.248235, 0}, {-1, 0.267778, 0}} *)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.