Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I have very big dataset, with over 14 000 000 records. And maybe more! Each record consists of Data ( Real ) and Time ( integer ).

I group records by time interval, resulting in 300 groups. In each group there are 70 000 records.

I need to draw a BoxWhiskerChart.

BUT, The system has frozen for 15 minutes. And then work is very slow. I think the reason this froze, is that I give it groups that are very large. But what can I do?

share|improve this question
1  
Welcome to SE Mathematica, Andrey. Could you specify where your code freezes? Could you load the data and Mathematica only fails plotting the chart, or you couldn't even import it successfully? Some information on how to load big data is here. –  István Zachar Apr 18 '12 at 10:05
1  
Andrey, can I also suggest that you register? Any future posts will then be linked properly to each other. –  Verbeia Apr 18 '12 at 11:13
add comment

1 Answer

Even on my rather old laptop, calculating the various statistics used for plotting the whisker of data sets of that size takes less than a minute so I suspect the problem is the number of groups rather than the number of records in each group. For each box in your plot, Mathematica generates a Tooltip containing a DynamicsBox. With 300 boxes, this will slow down the front end quite a lot.

It's a bit more involved than using BoxWhiskerChart out of the box, but as an alternative you could build a chart by hand using graphics primitives, for example

plot[data_] := DynamicModule[{n, min, max, q25, med, q75, fun},
  n = Length[data];
  {min, q25, med, q75, max} = 
   Transpose[
    Through[{Min, Quantile[#, 1/4] &, Median, Quantile[#, 3/4] &, Max}[#]] & /@ data];
  fun[dat_] := Transpose[{Range[n], dat}];
  Dynamic[Tooltip[Graphics[{
        {LightGray, Polygon[Join[fun[min], Reverse@fun[max]]]},
        {LightBlue, Polygon[Join[fun[q25], Reverse@fun[q75]]]},
        {Thick, Black, Line[fun[med]]},
        {Gray, Line[{{#, min[[#]]}, {#, max[[#]]}}]},
        {Thick, Blue, Line[{{#, q25[[#]]}, {#, q75[[#]]}}]}}, 
       Axes -> True, AspectRatio -> 1/GoldenRatio],
      Grid[{{"Index", #}, {"Max", max[[#]]}, {"75%", 
         q75[[#]]}, {"Median", med[[#]]}, {"25%", q25[[#]]},
        {"Min", min[[#]]}}, Frame -> All]] &@
    Clip[Round[MousePosition["Graphics"] /. None -> {1}][[1]], {1, n}]]]

data = Table[RandomVariate[NormalDistribution[i, i^2], 1000], {i, .1, 1, 1/300}];

plot[data]

screenshot

Here, I've chosen to plot the boxes and whiskers as polygons since with 300 boxes side-by-side it's hard to distinguish the individual boxes anyway.

share|improve this answer
1  
Using PerformanceGoal -> "Speed" as an option to BoxWhiskerChart will stop the tooltips. –  wxffles Apr 18 '12 at 21:18
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.