# Why is AllTrue much slower than VectorQ?

This seems to be a theme with V10, a new function dedicated to a specific task doesn't live up to expectation performance-wise. Mr. Wizard has already uncovered 2 such functions here and here. So how about AllTrue versus VectorQ? From the docs

We're interested in the second usage of VectorQ here, which is the same as the purpose of AllTrue. A quick test shows that VectorQ annihilates AllTrue when it comes to performance e.g.

Needs["GeneralUtilities"]

AccurateTiming[AllTrue[Range[10^6], IntegerQ]]


0.20069455

While for VectorQ:

AccurateTiming[VectorQ[Range[10^6], IntegerQ]]


0.00214923153

That is a two order of magnitude performance increase over AllTrue. Using BenchMarkPlot from the useful GeneralUtilities package we observe the following:

BenchmarkPlot[
{VectorQ[#, IntegerQ] &, AllTrue[#, IntegerQ] &},
Range[#] &,
PowerRange[10, 10^8],
"IncludeFits" -> True, PlotRange -> Full
]


So, why is AllTrue slow compared to VectorQ?

-
It is clear both from the absolute timings, and the complexity, that VectorQ[..., IntegerQ] has been specially overloaded on packed arrays to be constant time. In other words, the top-level evaluator doesn't actually evaluate IntegerQ on every element in a list here. But AllTrue has no choice, being more general function, and has a linear complexity, as it should. –  Leonid Shifrin Jul 19 '14 at 23:11
@LeonidShifrin. Thanks that explains it. Do you want to post that as an answer or should I delete the question? –  RunnyKine Jul 19 '14 at 23:13
Re: post vs delete - let's see what others think. Actually, this issue has been already discussed in another place here on SE (as a sub-topic of a wider discussion), but I can't find it now. –  Leonid Shifrin Jul 19 '14 at 23:19
Note that Range[10^6] is slower than VectorQ. You might want to do the comparison with the range stored in a variable, data = Range[10^6]. –  Michael E2 Jul 20 '14 at 0:17
@MichaelE2, I don't understand. I'm not comparing Range to VectorQ. –  RunnyKine Jul 20 '14 at 0:19

I guess I'll answer my question. As Leonid Shifrin alluded to in the comment, AllTrue is actually a more general function than VectorQ and this has various consequences:

(1) VectorQ is overloaded to work efficiently with packed arrays which is evident from my benchmark above. (2) If we append a non-integer to the example in the question, we no longer have a packed array and the complexity of both functions are now the same O(n):

BenchmarkPlot[
{VectorQ[#, IntegerQ] &, AllTrue[#, IntegerQ] &},
Append[Range[#], 3.4] &,
PowerRange[10, 10^8],
"IncludeFits" -> True, PlotRange -> Full
]


It appears that VectorQ is still slightly faster than AllTrue maybe someone can shed more light on this scenario. (3) AllTrue` returns symbolic results if test is applied to a symbol, this must lead to more overhead from such symbolic preprocessing.

-
I think the growing wisdom is: wait until version 10.0.2 to upgrade. –  Mr.Wizard Jul 20 '14 at 2:31
@Mr.Wizard. I agree. It's looking like v10 was released before it was ready for primetime. –  RunnyKine Jul 20 '14 at 4:21
I can't plot the graph like yours with your code now and my mma version is 10.1.Why?Can you help me?Thank you. –  WateSoyan May 8 at 2:10