2
$\begingroup$

MMA 11.0.1.0 Win 10 64-bit i8700K

The question is: how can one efficiently and speedily produce DateListPlots (or ListLinePlots with a date-time labelled x-axis calculated from x ordinates measure in seconds) with coordinate tooltips for lists of 10^5-10^6 points (or more)

...ideally in a way that is compatible with @istvan-zachar's excellent PlotExplorer on this site.

Background and Commentary I have quite a fast PC at my disposal, and sometimes Mathematica is remarkably fast and efficient, and sometimes it is the opposite.

Given this simple function (for the benefit of other neophytes like me: experts here would probably do this all in one line, but this is clearer to me)

x = Most[Range[0, 1, 0.000001]];
w = 500;
h = IntegerPart[9 w/16];
k = 3;
y = Sin[2 Pi k  x];
AbsoluteTiming[
ListPlot[Transpose[{x, y}], ImageSize -> {w, h}, Joined -> True]]

MMA plots (a 3 cycle sine wave comprising) 1,000,000 points in 3.12s

Likewise (data omitted), DateListPlot[persistedData, ImageSize -> Large] takes only 5.3s for 200k points, i.e 8.5x longer per point, but the overhead of seconds to date conversion involved probably accounts for that. (See below for comment on "persisted" data.)

Unfortunately I want to be able to read values off the graph; and therefore plot with Tooltip as, e.g.

DateListPlot[
 Tooltip[#,  
    DateString[#[[1]], stdDateTimeFormat] <> ", " <> 
     ToString[ #[[2]]]] & /@ 
  persistValues[
   Take[persistedData, Min[{Length[persistedData], 5000}]]], 
 ImageSize -> Large, Joined -> True, 
 PlotMarkers -> {{Graphics@Disk[]}, 0.001}]

where

stdDateTimeFormat = {"Day","/","Month", "/", "Year", " ", "Hour", ":", "Minute", ":", "Second", ".", "Millisecond"};

and my data is {{time1, value1},{time2, value2},...}

Note that I used Disk[], for which the timing increases polynomially

1000 points - 1.8s
2000 points - 7.1s
5000 points - 29.2s

Using PlotMarkers->Automatic, 5000 points could be plotted in 9.9s, but the size of the PlotMarkers is too great; PlotMarkers->{Automatic, Tiny} took 16.3s (and PlotMarkers->{Automatic} took ~15s)

The problem seems to be that if Tooltips are required, the need for plot markers causes a dramatic slowdown to unusability.

My x-axis data is available either in milliseconds since the 1-1-1900 00:00 or a formatted datetime string; I convert the former (without units) to dates with

msToDateTime[ms_]:=DatePlus[epochStartDateObject,{ms/1000,"Second"}];

About "persisted" Data

Standard line plots join successive points; it is conveniently assumed that the points are effectively samples and that changes can be interpolated - by straight lines, splines etc.

However, if the data to be plotted is not a sample but the population itself, the only changes are those in the data and the lines should be horizontal between consecutive points. The following function "persists" a value until the next change, turning

enter image description here

Into this

enter image description here

(* When plotting a time series such as financial data where the value persists until the next change, the simple line join of list plot misleads; so, we make each value persist until the next change *)
persistValues::usage = "Takes a list of coordinate pair lists and adds additional points that repeat the previous value at the subsequent ordinate, making all lines horizontal or vertical";
persistValues[xy_List]:=Partition[Flatten[{Table[{xy[[i]],xy[[i+1]][[1]], xy[[i]][[2]]},{i,1,Length[xy]-1}],Last@xy}],2]

Optional supplementary question: how would I define a FlatLinePlot function in terms of the above and e.g. ListLinePlot so that it inherited all ListLinePlot's OptionValues and passed any provided on? I wasn't able to work that out.

$\endgroup$
  • $\begingroup$ for FlatLinePlot lookup ListStepPlot and DateListStepPlot $\endgroup$ – kglr Oct 20 '18 at 15:45
  • $\begingroup$ ...also InterpolationOrder->0 $\endgroup$ – Mike Honeychurch Oct 21 '18 at 0:31
  • $\begingroup$ @kglr I can't believe I didn't find ListStepPlot in the documentation... but I see now that it isn't in the See Also list below ListPlot. Thank you; I console myself with the thought that I learned something by reinventing the wheel. (DateListStepPlot IS listed under Date list plot though). Will check out performance later. Thanks again. $\endgroup$ – Julian Moore Oct 21 '18 at 7:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.