Counting variable from a list of matrices

Question

I have a list of matrices

data= {{{23,3,4,5,20.5},{24,3,1,0,20.5},{25,3,7,8,20.5},{26,6,5,4,20.5}},{{23,4,5,3,20.4},{24,4,3,5,20.4},{26,4,3,2,20.4},{27,4,5,7,20.4},{28,4,3,2,20.4}},{{23,4,5,3,20.3},{24,4,3,5,20.3},{26,4,3,2,20.3},{27,4,5,7,20.3},{28,4,3,2,20.3},{29,0,0,2,20.3}},{{23,4,5,3,20.2},{24,4,3,5,20.2},{26,4,3,2,20.2},{30,4,5,7,20.2},{29,0,0,2,20.2}}}

Each matrix represents a collection of particles at a given time, each row represents a particle with its information {partice ID, x,y,z, time}, the $(x,y,z)$ coordinates can be ignored for this problem. Particles can merge together and form a new particle, in which case the ID of the more massive particle is retained on the merged particle.

I want to define a variable which counts the total number of NEW particle at each time. So in the given example, at time 20.5, I have 4 particles, at t=20.4 I have 2 new particles emerging, so now I have 6 particles at 20.4, at 20.3 I have 1 new halo emerging, so I have 7 particles, and again at 20.2 I have 1 new halo and so I have 8 particles. I want to have the final result as a table

{{20.5,4},{20.4,6},{20.3,7},{20.2,8}}

How do I do this? Thanks

EDIT : I was imagining, something like the Tally function, but finds the distinct particle that arise at each time, so tally compared to the previous group of particles.

EDIT 2: The particle IDs are random numbers.

Answer 1

When you take the IDs of each time-step and accumulate them through the time (but deleting duplicates), then you get the information you want. Therefore, the basic idea is to extract all IDs from each time and then you use Union over and over again trough all time-steps

FoldList[Union, data[[All, All, 1]]]
(* 
   {{23, 24, 25, 26}, 
    {23, 24, 25, 26, 27, 28}, 
    {23, 24, 25, 26, 27, 28, 29}, 
    {23, 24, 25, 26, 27, 28, 29}, 
    {23, 24, 25, 26, 27, 28, 29, 30}} 
*)

When you look at the lengths of each item, you see that it contains the number you are interested in.

I'm not sure why you chose this kind of representation of your data, because it is not really handy to work with, but your final answer can be extracted by

Transpose[
 {
  First /@ data[[All, All, -1]],
  Length /@ FoldList[Union, data[[All, All, 1]]]
  }
 ]
(* {{20.5, 4}, {20.4, 6}, {20.3, 7}, {20.3, 7}, {20.2, 8}} *)

Btw, it seems you have a double entry in your data.

Answer 2

Another approach combining @SquareOne's observation with ArrayComponents:

counts = Max /@ ArrayComponents[data[[All, All, 1]]] 
(* {4, 6, 7, 8} *)

times = First /@ data[[All, All, -1]];
Transpose[{times, counts}]
(* {{20.5, 4}, {20.4, 6}, {20.3, 7}, {20.2, 8}} *)

Note: Assumed that 0 is not used as a particleID.

Answer 3

I think this does what you need:

With[{a = Merge[Apply[#5 -> #1 &, data, {2}], Union]},
 Thread[{Keys@a, Length /@ FoldList[Union, Values@a]}]]

(* {{20.5, 4}, {20.4, 6}, {20.3, 7}, {20.2, 8}} *)

Answer 4

Slightly different approach. This

Max[#[[All, 1]]] & /@ data 

{26, 28, 29, 29, 30}

gives you the maximum particle number ID at each successive time (notice as @Halirutan said that you have a double entry (29), but never mind)

Theoretically, by calculating the successive differences between theses elements:

Max[#[[All, 1]]] & /@ data // Differences

{2, 1, 0, 1}

you get the number of new particles at each time:

Now, you just need to compute the cumulative sum of this, adding the initial number of particles is which is:

Length@First@data

4

so:

newp = FoldList[Plus, Length@First@data, Max[#[[All, 1]]] & /@ data // Differences]

{4, 6, 7, 7, 8}

Finally, each successive time in your data is given by:

times = data[[All, 1, -1]]

{20.5, 20.4, 20.3, 20.3, 20.2}

Finally

Thread[{times, newp}]

{{20.5, 4}, {20.4, 6}, {20.3, 7}, {20.3, 7}, {20.2, 8}}