4
$\begingroup$

Turn on

Needs["CompiledFunctionTools`"]

First example

Compile[{{x, _Real, 2}, {y, _Real, 1}}, 
  Module[{tmp}, tmp = x.y]] // CompilePrint

shows

        2 arguments
        1 Integer register
        3 Tensor registers
        Underflow checking off
        Overflow checking off
        Integer overflow checking on
        RuntimeAttributes -> {}

        T(R2)0 = A1
        T(R1)1 = A2
        I0 = 12
        Result = T(R1)2

1   T(R1)2 = Dot[ T(R2)0, T(R1)1, I0]]
2   Return

No copytensor, result of Dot directly assigned to tensor register T(R1)2

Second example,

Compile[{{x, _Real, 2}, {y, _Real, 1}}, 
  Module[{tmp = {1., 2.}}, tmp = x.y]] // CompilePrint

shows

        2 arguments
        1 Integer register
        5 Tensor registers
        Underflow checking off
        Overflow checking off
        Integer overflow checking on
        RuntimeAttributes -> {}

        T(R2)0 = A1
        T(R1)1 = A2
        I0 = 12
        T(R1)2 = {1., 2.}
        Result = T(R1)3

1   T(R1)3 = CopyTensor[ T(R1)2]]
2   T(R1)4 = Dot[ T(R2)0, T(R1)1, I0]]
3   T(R1)3 = CopyTensor[ T(R1)4]]
4   Return

There is two CopyTensor. It seems that mma detected that tmp is changed later. So it first copied to new tensor register T(R1)3, and assign Dot result to new tensor register T(R1)4, finally copy T(R1)4 to T(R1)3.

But this doesn't make sense to me. These two CopyTensor are unnecessary! Why not just assign the Dot result directly to T(R1)2 the same as the first example?? For safety reason? If so, is there any hidden option of compile could turn this safety off, if programmer is sure about what he is doing?

PS: Also there is strange assignment I0=12 used in Dot, what does it mean?

This kind of copying Tensor will affect performance sometimes, for example see here


Now let's see how C++ Eigen library deals with it

Eigen takes copying issue seriously, it provides .noalias() to address this problem. There is an elaborate table in the documentation titled "Writing efficient matrix product expressions " which lists all common situations that noalias can improve performance, I copied several case here enter image description here enter image description here

$\endgroup$
  • $\begingroup$ I see no difference in timing. I wonder what, if anything, is the effect of CopyTensor here. $\endgroup$ – Michael E2 Jan 7 '16 at 4:08
  • $\begingroup$ @MichaelE2 What if they are in a loop, and there are many such dot and assignment? $\endgroup$ – matheorem Jan 7 '16 at 5:10
  • 1
    $\begingroup$ But they aren't in a loop. I'm only saying that your two examples give roughly the same timing with RepeatedTiming for various size inputs. Sometimes the first is a little faster, sometimes the second. $\endgroup$ – Michael E2 Jan 7 '16 at 12:30
  • $\begingroup$ @MichaelE2 I don't know how to understand why my example shows no efficiency problem, but copytensor do affect efficiency, eg. stackoverflow.com/q/8183501/1911722 . and also I am learning Eigen now, it seems that they have taken copytensor issue seriously and offer specific options like noalias() to avoid copytensor, see eigen.tuxfamily.org/dox/group__TutorialMatrixArithmetic.html under item "Matrix-matrix and matrix-vector multiplication" $\endgroup$ – matheorem Jan 10 '16 at 15:28
  • 1
    $\begingroup$ I don't know how the WVM works, but there are (at least) two ways to execute the CopyTensor in your code. The cheap way is to make T(R1)3 point to the tensor (and copy the memory values later, if necessary, but it's provably unnecessary in this program). The expensive way is to blindly copy the memory to a new location. The timings suggest it is not doing the expensive way in this case. It could also be CPU dependent, for all I know, but I'm assuming you get the similar timings. Can you find a simple example of an unnecessary CopyTensor that impacts timing? $\endgroup$ – Michael E2 Jan 10 '16 at 15:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.