13
$\begingroup$

Take the following example:

big = RandomReal[1, 10000000];
Head[big >= 0.66] // AbsoluteTiming

(* {0.53489, GreaterEqual} *)

Why does this take a long time to evaluate, given that the expression doesn't actually change?


Update: Simon Woods points out that the slowdown is due to unpacking. There is also some strange interaction with ReplaceAll that I do not understand. Both are demonstrated by the following:

r=RandomReal[1,10000000];

On["Packing"]

Hold[Greater[r,0.5]]/.{x_?ArrayQ:>x}//AbsoluteTiming
Developer`FromPackedArray::unpack: Unpacking array in call to Greater.
{0.497073,Hold[r>0.5]}

Hold[greater[r,0.5]]/.{x_?ArrayQ:>x}//AbsoluteTiming
{0.000017,Hold[greater[r,0.5]]}

ArrayQ[r]//AbsoluteTiming
{3.*10^-6,True}

Note:

  • big is a large packed array
  • big >= 0.66 is the same as GreaterEqual[big, 0.66]
  • the result is identical to GreaterEqual[big, 0.66]

This is a problem for my BoolEval package, which has a BoolEval function that transforms the expression big >= x to UnitStep[big - x] to speed up element-wise comparisons. big >= x takes longer to evaluate than UnitStep[big - x].

Here's a much simplified version of the function:

greatereq[a_, b_] := UnitStep[a-b]
BoolEval[expr_] := expr /. GreaterEqual -> greatereq

A potential solution is

SetAttributes[BoolEval, HoldAll]
BoolEval[expr_] := Unevaluated[expr] /. GreaterEqual -> greatereq

The problem with this is that it will not work with

f[] := big >= 66
BoolEval[f[]]

Such a function would simply be too confusing to use.

$\endgroup$
  • $\begingroup$ @Kuba The actual function can handle arbitrary Boolean expressions such as 0.1 < arr < 0.5 && arr != 0.4. This solution would hide the performance problem for a few common cases, but otherwise it will still be slow. So I'm not fully happy with it. $\endgroup$ – Szabolcs Jul 12 '16 at 13:30
  • 2
    $\begingroup$ On["Packing"] reveals that the array is unpacked in the call to GreaterEqual $\endgroup$ – Simon Woods Jul 12 '16 at 20:21
  • 1
    $\begingroup$ How about the HoldAll version but instead of Unevaluated use Block[{GreaterEqual}, ...] $\endgroup$ – Simon Woods Jul 12 '16 at 20:25
  • 2
    $\begingroup$ @SimonWoods I thought of it but the difficulty with that is the following: What if someone does BoolEval[myFun[] > 1] and myFun[] uses Greater? I am probably going to do HoldAll and require the boolean expressions and comparisons to appears explicitly in the argument. I.e. BoolEval[x > 1] is okay, but fun[]:=x>1; BoolEval[fun[]] is not. However, I am not willing to break BoolEval[myFun[] > 1] as well. $\endgroup$ – Szabolcs Jul 13 '16 at 8:29
  • 1
    $\begingroup$ @Mr.Wizard I more or less gave up on this question, partly because WRI explained why there's unpacking (and it's unlikely to go away), and partly because I don't think I was taking the correct approach anyway. You are right about the ReplaceAll—embarrassing oversight. I don't think Block will be a good approach, but too little space to explain here ... I wanted to entirely rewrite BoolEval and blog about it. We can discuss next week in chat if you're interested, otherwise I'll ping you when the blog is out. $\endgroup$ – Szabolcs Feb 26 '17 at 19:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.