Major slowdown when using a ColorFunction with more than one named scheme

Question

Bug introduced in V10.1 or earlier and persists through 11.2

---

Earlier Kuba noted a major slowdown with code Simon Woods posted. Here is a simplified example.

First a color function using two different named gradient schemes:

fn = ColorData[If[# < 0.5, "CoffeeTones", "DeepSeaColors"]][#] &;

Now its use to colorize a random array:

Colorize[
 RandomInteger[99, {100, 100}],
 ColorFunction -> fn
] // AbsoluteTiming

This is conspicuously slow for such a simple operation. Interestingly this has nothing to do with Colorize or how it uses fn; it is inherent to fn itself:

fn /@ RandomReal[1, 100^2] // AbsoluteTiming // First

4.746

What's going on?

Answer 1

It seems that there is a significant overhead every time a color scheme is switched. Once a scheme is loaded each use is fast, but changing color schemes apparently unloads and reloads the mechanism. The result is that the speed of application is directly related to the frequency of switching.

With sorted values there is only one switch and application is fast:

fn /@ Sort @ RandomReal[1, 100^2] // AbsoluteTiming // First

0.014

The same is observed within Colorize:

Colorize[
  Range[100^2] ~Partition~ 100,
  ColorFunction -> fn
] // AbsoluteTiming

With semi-frequent switches the performance is in between:

fn /@ Rescale[Range[100^2] ~Mod~ 100`] // AbsoluteTiming // First

0.234

One can get a sense of the magnitude of the scheme reload procedure with this:

ColorData["DeepSeaColors", 0.7];  (* preload DeepSeaColors *)
ColorData["CoffeeTones", 0.1];    (* preload CoffeeTones   *)

tr1 = Trace[ColorData["CoffeeTones", 0.3], TraceInternal -> True];

tr2 = Trace[ColorData["DeepSeaColors", 0.7], TraceInternal -> True];

ByteCount /@ {tr1, tr2}

{519512, 10407576}

This shows that the Trace for a reload is ten megabytes in length! And this is after previously using both color schemes. So this huge procedure is called every time a different color scheme is used; no wonder it's incredibly slow!

It seems to me that this massive overhead is a bug, plain and simple. My evidence is that using {sceme, "Reverse"} avoids the problem. The use of "Reverse" causes the raw color data to be used in Blend (rather than the scheme name) as I once discussed with J.M.. Note that alternating color schemes are still being called by name -- only the "Reverse" parameter is added.

fn2 = ColorData[{If[# < 0.5, "CoffeeTones", "DeepSeaColors"], "Reverse"}][1 - #] &;

This function performs as we might expect the original one to:

A Trace shows that the massive overhead is avoided:

ColorData[{"DeepSeaColors", "Reverse"}, 0.7];  (*preload DeepSeaColors*)
ColorData[{"CoffeeTones", "Reverse"}, 0.1];    (*preload CoffeeTones*)

tr3 = Trace[ColorData[{"CoffeeTones", "Reverse"}, 0.3], TraceInternal -> True];

tr4 = Trace[ColorData[{"DeepSeaColors", "Reverse"}, 0.7], TraceInternal -> True];

ByteCount /@ {tr3, tr4}

{898696, 897736}

This is still quite a long Trace for such an operation. We can avoid it if we use the raw color scheme data as Kuba did in answering the question that prompted this one, however for this example it can be done more simply.

The raw color data is available by using the Property "BlendArgument":

Head /@ ColorData["CoffeeTones", "BlendArgument"]

{RGBColor, RGBColor, RGBColor, RGBColor, RGBColor, RGBColor}

We can simply construct or our Blend from this data and use it instead:

mem : gradient[name_String] := mem =
  ColorData[name, "BlendArgument"] /. x_List :> (Blend[x, #] &)

Just replace ColorData with gradient:

fn3 = gradient[If[# < 0.5, "CoffeeTones", "DeepSeaColors"]][#] &;

 Colorize[RandomInteger[99, {100, 100}], ColorFunction -> fn3] // AbsoluteTiming

Prestidigitonium!