Quickly plotting (number of events) vs. (time) for a very large number of events encoded as pairs of real numbers representing start and end times

I have an array that looks like the following:

exampleArray = {{t1,r1},{t2,r2},{t3,r3},{t4,r4},{t5,r5},...};


Where each pair of real numbers $(t_k,r_k)$ represents some "event" (independent of all other events) starting at time $t_k$ and ending at time $r_k$. Provided some time $t_i$ and exampleArray, I'd like to quickly return the number of "events" at this time. I'd also like to quickly plot the number of events versus time over some interval. The catch is that exampleArray might have a very large number of elements, so we wouldn't want to scan through it multiple times.

Is there an elegant way to do this in Mathematica?

First, make up some data in the form of the exampleArray:

startTimes = RandomReal[{0, 100}, 50];
endTimes = startTimes + RandomReal[{0, 10}, 50];
ex = Transpose[{startTimes, endTimes}]


Here's a function to find how many of the pairs straddle the value t:

sel[t_] := Length[Select[ex, #[[1]] < t < #[[2]] &]]


For example, calling sel[25] counts how many of the pairs contain the point t=25. Calling

sel[#] & /@ {25, 30, 35}


shows how many times 25, 30 and 35 are contained. You can plot straightforwardly:

ListPlot[Table[sel[t], {t, 1, 100}]]


• Very cool... can we plot with this function? – user11745 Jan 14 '14 at 19:14

It's certainly not as fast as the implementations already provided, but have you considered looking at EventData and the related functionality introduced in M9?

Using Kuba's data:

startTimes = RandomReal[{0, 100}, 100];
endTimes = startTimes + RandomReal[{0, 10}, 100];
events = EventData[Transpose[{startTimes, endTimes}]];


Use SurvivalModelFit to build a survival model of the data, this is like LinearModelFit in that it creates an object with properties. The useful one here is "EventMatrixPlot"

AbsoluteTiming[SurvivalModelFit[events]["EventMatrixPlot"]]


data = [Sort /@ RandomReal[100, {100, 2}];


Now let's get list of beginings and endings:

{start,stop}=Transpose[data]


Now both time-start[[i]] and stop-time[[i]] have to be positive:

sel2[t_] := Total[UnitStep[t - start] UnitStep[stop - t]]

Plot[Total[UnitStep[t - start] UnitStep[stop - t]], {t, 0, 100}]


Speed comparison:

startTimes = RandomReal[{0, 100}, 100];
endTimes = startTimes + RandomReal[{0, 10}, 100];

ex = Transpose[{startTimes, endTimes}];

sel[t_] := Length[Select[ex, #[[1]] < t < #[[2]] &]];
sel2[t_] := Total[UnitStep[t - startTimes] UnitStep[endTimes - t]]

n = 1000;
k = RandomReal[100];
sel[k]~Do~{n} // Timing
sel2[k]~Do~{n} // Timing

{0.312002, Null}
{0.015600, Null}

• seems to be ~10x faster. – Kuba Jan 14 '14 at 19:39
• I apologize, but could you explain your answer a little bit? What is causing the speedup in your opinion? (NOT a criticism, I'm just being slow) – user11745 Jan 14 '14 at 20:24
• @user11745 plotting may seem slow because it's "continuous" plot, not ListPlot for 100 values. sel2 is faster because there are no logical tests, only matrices manipulations. I will update it in a minute with speed comparison. – Kuba Jan 14 '14 at 20:29
• @user11745 is it clear now? ;) – Kuba Jan 15 '14 at 6:09