62
$\begingroup$

I installed Mathematica 12.1 a few days ago and now I experience a serious speed slow down. When working with datasets for example Mathematica is close to be unusable. Even in the help systems it lasts more than a second until a reactions comes. Also scrolling through a dataset is nearly impossible, even for small datasets (460 x 53).

Is here anyone with similar problems and can help me out. For the moment I changed back to 12.0 which is much more performant.

I use a MacMini with 6 cores, 64GB of Ram and 2 TB SSD, actual MacOS

$\endgroup$
18
  • 2
    $\begingroup$ Also, which specific Mac OS is being used? This information could be useful for diagnosis at our end. $\endgroup$ Mar 21, 2020 at 17:21
  • 12
    $\begingroup$ @DanielLichtblau I thought this bug was very well known internally, at least several of us beta testers complained before release ... Please do make a 12.1.1 which corrects this. Just playing with Wolfram's version of the epidemic data, a relatively tiny dataset, is very painful (while 12.0 works fine) $\endgroup$
    – Szabolcs
    Mar 21, 2020 at 19:27
  • 1
    $\begingroup$ Is there a specific help page that exhibits this slowdown? $\endgroup$ Mar 21, 2020 at 19:37
  • 1
    $\begingroup$ @DanielLichtblau: You are absolutely right I wrote only "actual MacOS" which is 10.15.3 - it would have been better to write it down directly. $\endgroup$
    – mgamer
    Mar 22, 2020 at 16:09
  • 1
    $\begingroup$ The frontend slowdown is not only limited to Dataset issues. It also hangs for crowded plots. Windows 10 version here. Unusable for me a.t.m. $\endgroup$ Mar 25, 2020 at 11:51

2 Answers 2

40
+150
$\begingroup$

Official Statement

In case some of you missed it: There was an official reply from one of the WRI devs recently in our chat

Hi. This is Jose, from Wolfram. We are aware of an unacceptable slowdown in some Dataset expressions, due to a bad dynamic interaction with the summary boxes of some objects. Both the TimeSeries objects of the coronavirus datasets and the InterpolatingFunction objects in the example given before typeset via summary boxes. We are working on a solution and will release it as soon as it is available.

At this point, Szabolcs answered

Thanks for the update. The example I posted does not have any summary boxes though, and it still makes the FE quite literally unusable. Import["https://covid.ourworldindata.org/data/ecdc/total_cases.csv", "Dataset", HeaderLines -> 1]

and the reply was

Many thanks for this example, @Szabolcs. We are looking into it too, and hope to have a fix very soon. We are improving Dataset in multiple ways, and WL 12.1 has focused on increased interactivity, Grid-like styling capabilities, storage of data in place, copy-paste, and other FrontEnd-related things. Summary boxes and similar typesetting constructs are important to identify and understand expressions, but they should not make the system much slower, of course.

Let's hope for the best.

Original Answer

Not an official answer nor a solution. Just some insight. I reported this issue about 4 months ago for the pre-release because I experienced considerable slowdowns when working with things that are "nicely rendered" in the front end. This includes dates, time-series, interpolating functions, etc. The issue is particularly noticeable when you have a data-set (in itself a formatted table with dynamic stuff) that contains things like TimeSeries.

My go-to example is

res = ResourceData["Epidemic Data for Novel Coronavirus COVID-19"]

This displays 20 rows and 7 columns and breaks my Linux and macOS front end. At home I have a Intel i7 Extreme with 8 cores (32GB RAM) and my macOS is running on the latest iMac Pro with 32GB RAM and the largest CPU available.

In particular, I saw the following issues:

  • hovering the header entries so that the links turn blue takes about 1 second
  • scrolling the notebook is awful when you hover your mouse over one the time-series entries
  • clicking the small arrow at the lower left to display the next 20 entries takes several seconds
  • any editing in the notebook has a lag and particularly selecting something with the mouse becomes impossible
  • resizing the window takes considerable more time

During some of these actions, I see the small "Progress Dialog" popping up.

Having said all that, we get an idea what exactly is the bottle-neck: front end rendering. So here is an example of 10x20 time-series placed in a dataset. This highlights the issue and I needed to delete the output cell just to copy the code here:

v = {2, 1, 6, 5, 7, 4};
t = {1, 2, 5, 10, 12, 15};
ts = TimeSeries[v, {t}];
Dataset[ConstantArray[
  Association @@ Table["Key" <> ToString[i] -> ts, {i, 10}]
  , 20]
 ]

Maybe some folks can try to reproduce the issue with the examples I've given.

Details

I will report this issue once more. I came up with the following demonstration, where we first create a dataset without showing it

head = CharacterRange["A", "E"];
ip[] := ListInterpolation /@ RandomReal[{-1, 1}, {Length[head], 10}];
ds = Dataset@Table[Inner[Rule, head, ip[], Association], {40}];

This takes about 0.01 seconds on my machine. Next, we look at the OutputForm of ds. Noteworthy, it is instantaneous although it shows the entire dataset. Furthermore, the front end is as fast as it should be

OutputForm[ds]

Next, we let the front end render the dataset in all its beauty

ds

This takes over 10 seconds, pops up a "Progress Dialog" and the front end is mostly unresponsive afterward.

Note: For me, it makes a difference if my notebook is fullscreen or not. If I have a notebook of normal size, where the dataset is cropped, it is much slower than in fullscreen, where the entire dataset is visible. Weird.

$\endgroup$
10
  • 10
    $\begingroup$ Just for completeness: The issues described in the original question and your answer also occur with the Windows frontend. $\endgroup$
    – Pirx
    Mar 21, 2020 at 21:22
  • 3
    $\begingroup$ I had no chance to test on Windows and I wasn't sure if it really showed up to that degree for the "flagship operating system". But thanks for testing! $\endgroup$
    – halirutan
    Mar 21, 2020 at 21:28
  • 1
    $\begingroup$ @halirutan: This is nearly the dataset I used (Data for Johns Hopkins, CSV transformed to `TimeSeries) ;-) $\endgroup$
    – mgamer
    Mar 22, 2020 at 6:58
  • 1
    $\begingroup$ Just a reminder that Query[] can be used on list of associations with good performance. I use Dataset the same way I use TraditionalForm: only for a final nice rendering when needed. $\endgroup$ Mar 27, 2020 at 15:55
  • 2
    $\begingroup$ @GustavoDelfino I never use Dataset. It's never been the most performant and I just don't need all this nice rendering during work. A list of associations is exactly what I use in almost all cases. $\endgroup$
    – halirutan
    Mar 27, 2020 at 18:14
27
$\begingroup$

Dataset was restructured in the 12.1 release in order to support expanded formatting options and interactivity such as hiding and sorting. As a result, some Dataset outputs showed a slowdown due to inefficiencies in the dynamic output structures they produce.

Because the code of Dataset is automatically field upgradable, we have released an update to its paclet that should fix those issues. Part of the fix involves simplifying the formatting of TimeSeries and other "summary boxes" in Dataset output by removing "+" opener and icon.

If you have version 12.1 installed and have used it recently, Dataset has probably by now been upgraded. You can check for an upgrade by evaluating

Information[PacletObject["TypeSystem"]]["Version"]
(* "12.1.0.2" *)

A version number of 12.1.0 means the update has not yet occurred. A version number of 12.1.0.2 means that the update has been installed on your system.

If your system has not yet automatically updated, you can force an update by evaluating this:

PacletSiteUpdate /@ PacletSites[]
PacletInstall["TypeSystem"]

With this update, Dataset should be no slower than in version 12.0, and in some cases faster.

$\endgroup$
8
  • 12
    $\begingroup$ Or the simpler PacletInstall["TypeSystem", UpdatePacletSites -> True]. Thanks for the quick fix! $\endgroup$
    – Szabolcs
    Apr 6, 2020 at 15:55
  • 1
    $\begingroup$ I already upgraded to 12.1.0.2 ..... better speed, but not quite. It seems Mathematica is becoming too big, it has too many things, AI and what have you.Video computing now. The new resolution great! Maybe Wolfram should sell an ultra fast MiniMathematica without the non mathematical stuff. $\endgroup$
    – Roberto
    Apr 7, 2020 at 5:56
  • 5
    $\begingroup$ in general Dataset is rather slow and can't be relied on for practical job: sluggish and unresponsive at most times. In comparison, pandas in the python world is such a relief. Hopefully, Dataset can be improved to a usable level as I can see many brilliant ideas behind this design and I do love Mathematica. $\endgroup$
    – sunt05
    Apr 22, 2020 at 12:35
  • $\begingroup$ Hi @jose, I'm revisiting this thread after watching your recent talk on Dataset improvement at WTC; the new type deducing system looks amazing! and you mentioned briefly it could be turned on for Dataset but I cannot find anything in v12.2. I'm just wondering if that's something available or we still need to wait. Anyway, really looking forward to the enhancement! $\endgroup$
    – sunt05
    Jan 21, 2021 at 20:18
  • 2
    $\begingroup$ Thanks for asking. In the end this was not included in 12.2 because it requires more testing. Sorry about that! $\endgroup$
    – jose
    Jan 25, 2021 at 21:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.