0
$\begingroup$

Was wondering what other options I have when encountering a time series that has two events taking place at the same time. In particular,

TimeSeries[{3, 8, 4, 11, 9, 2}, {{1, 1, 5, 6, 7, 10}}]

gives

{{1, 11/2}, {5, 4}, {6, 11}, {7, 9}, {10, 2}}

Where is this outlined in the documentation that we just average the values (3+8)/2 for time at 1 ?

I would like to have those two data points as distinct events. How can I go about that ?

And, how can I specify not to average those two datapoints but instead apply another function.

Specific Use-case

Assume timeseries, ts1, that contains timestamps with corresponding prices. Let's say for simplicity that timestamps are in seconds.

And assume that there are many events happening during one second (Buy Sell or BUY BUY, or BUY BUY BUY, etc ... at same or even different prices).
So, we could have

prices={10, 10,10.4, 10.6,10.7,11,12}
times={1,1,1,2,3,4,4}

Here we have 3 events happening in the first second, and two events happening in the 4th second.

Question

I can write my own piece of code to group these events and perform my own statistical analysis, but thought there's a more intelligent way to do these manipulations with TimeSeries (and since TimeSeries does unwanted things like taking averages [?!] without even warning the user - undocumented behavior! The online help browser for V12 does not list this behavior).

ts1=TemporalData[Transpose@{times, prices }]

Without further argument input does the same thing as TimeSeries

ts1["Path"]

outputs

{{1,10.1333},{2,10.6},{3,10.7},{4,23/2}}

Unless I specify something like

 ts2=TemporalData[Transpose@{times, prices }, Automatic, ValueDimensions -> 2]

Here I get only 7 datapoints, which is what I expect, but I am not making much sense of

  ts2["Path"]

{{0,{1,10}},{1,{1,10}},{2,{1,10.4}},{3,{2,10.6}},{4,{3,10.7}},{5,{4,11}},{6,{4,12}}}

a) How should I interpret the DateListPlot output for ts2

b) If I want to perform a TimeSeriesAggregation with parameters dt=1 second, and count the number of events happening during that one second. how should I specify that ?

TimeSeriesAggregate[ts2, 1, Length[#]&]

That surely doesn't work or make sense. I would expect an output like

{3,1,1,2}

Update:

TimeSeriesAggregate[ts2["Values"], 1, Length[#]&]

maybe getting closer, but then does work unless we have to literally specify times in terms of some Date format here, rather than integers.

With date format I think something more like what I need comes to shine, but it does funny things with the last data point ... like throwing it out without counting ... ??? So, below, I would like to specify that we are want to count events in non-overlapping 1 second intervals .... but what's happening ? how is dt=1 interpreted here ? Not the way the user expects it to ...

TimeSeriesAggregate[{{"2019-09-12 00:01:23",1}, {"2019-09-12 00:01:23", 3}, {"2019-09-12 00:01:24",10}, {"2019-09-12 00:01:25", 55}, {"2019-09-12 00:01:25", 10}}, 1, Length[#] &]
$\endgroup$
  • $\begingroup$ It's not mentioned as to how you might want to use the result. Doing so might suggest other approaches. If you just need pairs of {time, value}, you could use Transpose[{{1, 1, 5, 6, 7, 10}, {3, 8, 4, 11, 9, 2}}] which gets you {{1, 3}, {1, 8}, {5, 4}, {6, 11}, {7, 9}, {10, 2}}. $\endgroup$ – JimB Oct 9 '19 at 22:42
  • 1
    $\begingroup$ By reading your question update it looks like you are looking for the functionalities described here: "Parametrized event records data transformations". $\endgroup$ – Anton Antonov Oct 10 '19 at 12:16
  • $\begingroup$ @AntonAntonov : Nice but you keep using TimeSeries in your code, which basically automatically "truncates" events with same timestamps into one event. So, my question hasn't been answered fully. Let me ask differently, how do I group events over a non-overlapping window of dt=10 seconds, if the time series has timestamps expressed as DateStrings, where there are different events sometimes occurring with same DateStrings. (If we feed the data into TimeSeries[], then of course, those same DateStrings merge into one DateString[]. Ideally, we would not want this to happen.) $\endgroup$ – Curious Oct 11 '19 at 6:41
  • $\begingroup$ "Nice but you keep using TimeSeries in your code, which basically automatically 'truncates' events with same timestamps into one event." -- No, you do not understand what article describes. Good luck with your search for answers. $\endgroup$ – Anton Antonov Oct 11 '19 at 13:33
  • $\begingroup$ @AntonAntonov I do understand the article, but you have not provided a simple mwe for what I asked for. So, thanks. $\endgroup$ – Curious Oct 12 '19 at 13:03
4
$\begingroup$

"I would like to have those two data points as distinct events."

It seems that you want to use TemporalData or EventData. (Not TimeSeries.)

td = TemporalData[{Transpose[{{3, 4, 11, 9, 2}, {1, 5, 6, 7, 10}}], {{1, 8}}}]

Show[ListLinePlot[td, PlotStyle -> {{}, {Thickness[0.025]}}, PlotRange -> All], 
     ListPlot[td]]

enter image description here

$\endgroup$
  • $\begingroup$ thank you @anton-antonov but hold on, who says, that I know where in the time series the double or sometimes triple or quadruple datapoint with same 'time stamp' occurs ? You parsed {1,8} knowingly. I guess you could easily do TemporalData[Transpose@{{3, 8, 4, 11, 9, 2}, {1, 1, 5, 6, 7, 10}}] without picking out {1,8}. My question is, why is it necessary to parse the ts in terms {t_i,v_i} pair now ? $\endgroup$ – Curious Oct 10 '19 at 0:34
  • $\begingroup$ Also does TemporalData handle milliseconds timestamps (correctly) ? I know it converts formats of "year month day hour minutes seconds:milliseconds" --> AbsoluteTime with 3 significant figure accuracy. I don't know how accurate it is. But how do I correctly revert from that AbsoluteTime calculated value back to a timestamp with milliseconds. For example, FromAbsoluteTime[ absoluteTime ] only provides "year month day hour minutes seconds" . $\endgroup$ – Curious Oct 10 '19 at 0:43
  • $\begingroup$ TimeSeriesAggregate[ TemporalData[Transpose@{{3, 8, 4, 11, 9, 2}, {1, 1, 5, 6, 7, 10}}], 1, Length@# &]["Paths"] I expected an output with a count of 2. Namely the datapoint {{3,1},{8,1}} ... $\endgroup$ – Curious Oct 10 '19 at 0:52
  • $\begingroup$ @Curious "who says, that I know where in the time series the double or sometimes triple or quadruple datapoint with same 'time stamp' occurs ?" -- Your question does not convey the more general point of view you explain in the comments here, nor it has the specific examples and code you give. Maybe you should revise it... $\endgroup$ – Anton Antonov Oct 10 '19 at 2:51
  • $\begingroup$ thank for the opportunity to edit my question , please see above @anton-antonov, (specifically why the length[#] within dt=1 does not work.) $\endgroup$ – Curious Oct 10 '19 at 5:15

Not the answer you're looking for? Browse other questions tagged or ask your own question.