Error Using Sow in DensityHistogram with ColorFunction

To find the color values used in a plot, I was using Sow as shown here:

Short@Reap[Histogram3D[RandomVariate[BinormalDistribution[.5], 500],
ColorFunction -> (ColorData["Rainbow"][Sow[#1]] &)]
]


But I get an error if I simply replace Histogram3D by

Short@Reap[
DensityHistogram[RandomVariate[BinormalDistribution[.5], 500],
ColorFunction -> (ColorData["Rainbow"][Sow[#1]] &)]
]


Now I get the following message:

Transpose::nmtx :  "The first two levels of the one-dimensional list"
{{{-3.5,3.},{-3.5,3.},{0.,24.}},0, ... , 1} cannot be transposed.

Transpose::argt :  "Transpose called with 0 arguments; 1 or 2
arguments are    expected."


It's possible to work around this problem by doing without Sow and Reap as follows:

l = {};
{DensityHistogram[RandomVariate[BinormalDistribution[.5], 500],
ColorFunction -> (ColorData["Rainbow"][AppendTo[l, #1]; #1] &)],
N@l}


But it would be nice to know if there is an explanation for why Sow doesn't work in the DensityHistogram example whereas it works in Histogram3D. The same error appears if I leave out the Reap.

From the message it seems clear that a list of all PlotRange tuples is being sown at some point before the ColorFunction actually is put to use. I tried to suppress passing that kind of argument to Sow by modifying the ColorFunction to

ColorFunction -> (If[ListQ[#1], White,
ColorData["Rainbow"][Sow[#1]]] &)


but it had no effect. As Leonid pointed out in his answer, there must therefore be a different invocation of Sow outside the color function.

Maybe someone knows another way to suppress the invalid ColorFunction call that seems to trip up Sow. However, it doesn't seem to be obvious because when I don't use Sow as in my work-around based on AppendTo with a list l, there is no entry in the resulting list l that corresponds to the PlotRange tuples.

Edit

It seems that this is a problem specifically with DensityHistogram. I also noticed that the setting for FrameTicks is ignored by DensityHistogram. It can be restored by wrapping the plot in Show with the desired FrameTicks option.

None of these issues arise with the related function SmoothDensityHistogram.

Leonid pointed out what the probable cause for the Sow error is, but since wrapping the result in another Show, I would for now stick with the workaround I mentioned above: ditch Sow and Reap, and use AppendTo to collect the desired values in a list l using a CompoundExpression of the type AppendTo[l, #]; #.

-
Is it possible for you to create an example that doesn't require version 8? I do not have DensityHistogram or BinormalDistribution -- does it affect only DensityHistogram? –  Mr.Wizard Jun 14 '12 at 16:19
@Mr.Wizard Fortunately (or unfortunately for you?) I so far only found this problem with DensityHistogram. You could replace BinormalDistribution by NormalDistribution with Randomvariate getting argument {500,2}, but that won't change anything. Sorry - I wasn't trying to rub in your lack of version 8... if I find another case I'll post an update. –  Jens Jun 14 '12 at 16:23
"I wasn't trying to rub..." I didn't think you were, but I'm afraid I cannot help then. Looks like Leonid is on the case so that shouldn't matter. :-) –  Mr.Wizard Jun 14 '12 at 16:33
I would not worry too much about unprotecting in this case, since I wrapped it inside InternalInheritedBlock, making this effect local to the execution stack inside withTaggedReapSow dynamic environment. The rest of the system is unaffected by these modifications. –  Leonid Shifrin Jun 14 '12 at 20:53
Thanks for the clarification, that's definitely a useful trick. –  Jens Jun 14 '12 at 20:58

This looks like a defect in the implementation. Here is a simple test:

traced =
Trace[
DensityHistogram[RandomVariate[BinormalDistribution[.5], 500],
ColorFunction -> (ColorData["Rainbow"][#] &)]
];

FreeQ[traced,Sow]

False


What this means is that Sow is also used in the implementation, and apparently without tags, so when Reap is used in the implementation to collect the points, it also collects yours. Tags should have been used by the implementors to avoid such collisions.

And here is a fix:

ClearAll[withTaggedReapSow];
SetAttributes[withTaggedReapSow, HoldAll];
withTaggedReapSow[code_] :=
Module[{internalTag},
InternalInheritedBlock[{Reap, Sow},
Unprotect[Reap, Sow];
Sow[arg_] := Sow[arg, internalTag];
Reap[arg_] := Reap[arg, internalTag, #2 &];
code]];


and now:

withTaggedReapSow[
Short@Reap[
DensityHistogram[RandomVariate[BinormalDistribution[.5], 500],
ColorFunction -> (ColorData["Rainbow"][Sow[#1, "MyTag"]] &)],
"MyTag",
#2 &
]
]


gives an expected result.

-
Sounds correct. I actually tried using tags myself but that didn't fix it... –  Jens Jun 14 '12 at 16:27
@Jens The problem is not that you are not using tags, but that tags are not used in the implementation of DensityHistogram. So, Reap in there collects anything which was sowed, and that includes your points. –  Leonid Shifrin Jun 14 '12 at 16:28
Yes, I think you nailed it. –  Jens Jun 14 '12 at 16:29
@Jens I added a fix, have a look –  Leonid Shifrin Jun 14 '12 at 16:35
That definitely suppresses the error. Only one thing: now it seems that the internal Reap/Sow didn't manage to produce a plot frame (apparently that was its purpose). But that can be fixed by wrapping the plot in Show[..., Frame -> True] –  Jens Jun 14 '12 at 16:48