Skip to main content
Tweeted twitter.com/StackMma/status/878784921979977728
Mod Moved Comments To Chat
added 660 characters in body
Source Link
Fred
  • 347
  • 1
  • 7

I am dealing with a multiplication of two- and three- dimensional arrays. Since the arrays are large, I am compiling a function to deal with it. The code of the function, with a working example, is as follows:

(*array dimensions*)
pp = 30; nn = 3000; 

(*sample data*)
a = Table[1., {p, 1, pp}, {n1, 1, nn}, {n2, 1, nn}];
b = Table[1., {p, 1, pp}, {n1, 1, nn}];
c = Table[1., {p, 1, pp}, {n1, 1, nn}];
d = Table[1., {p, 1, pp}, {n1, 1, nn}];

(*defining compiled function*)
With[{P = pp}, (*localizing P to avoid MainEvaluate*)
   y=Compile[{{a, _Real, 3}, {b, _Real, 2}, {c, _Real, 2}, {d, _Real, 2}},
        Module[{num, den, i1, out},
             i1 = b c d ;
             num = a i1;
             den = Table[Total[num[[p]]], {p, 1, P}]; 
             out = Table[ 
                 Transpose[Transpose[num[[p]]]/den[[p]]], 
               {p, 1, P}];  
             out
        ] 
     ]
]; 
  • a: P pp x Nnn x Nnn
  • b,c,d: Ppp x Nnn

To be clear, the purpose of the last instruction in the compiled function is to:

  • take "page p" of num (I am thinking of a 3-dimensional array as a book, with pages, and tables NxN in each page);
  • divide each column by the correspondent row of den, to obtain again an NxN matrix (this is why I am using a double Transpose instruction).
  • create Ppp pages of such matrices.

In my application, P~30pp~30 and N~3000nn~3000. It takes about 109-10 seconds to run this function. on my (fairly fast) computer:

 DateString[]
 test = y[a, b, c, d];
 DateString[]

 Out[363]= "Thu 22 Jun 2017 09:10:43"

 Out[365]= "Thu 22 Jun 2017 09:10:54"

Since I will need to run it thousands of times, this is a pretty inefficient code. I am looking for a way to substantially speed up the computation, possibly avoiding the Transpose functions.

I am dealing with a multiplication of two- and three- dimensional arrays. Since the arrays are large, I am compiling a function to deal with it. The code of the function is as follows:

y=Compile[{{a, _Real, 3}, {b, _Real, 2}, {c, _Real, 2}, {d, _Real, 2}},
     Module[{num, den, i1, out},
          i1 = b c d ;
          num = a i1;
          den = Table[Total[num[[p]]], {p, 1, P}]; 
          out = Table[ 
              Transpose[Transpose[num[[p]]]/den[[p]]], 
            {p, 1, P}];  
          out
     ] 
 ]; 
  • a: P x N x N
  • b,c,d: P x N

To be clear, the purpose of the last instruction is to:

  • take "page p" of num (I am thinking of a 3-dimensional array as a book, with pages, and tables NxN in each page);
  • divide each column by the correspondent row of den, to obtain again an NxN matrix (this is why I am using a double Transpose instruction).
  • create P pages of such matrices.

In my application, P~30 and N~3000. It takes about 10 seconds to run this function. Since I will need to run it thousands of times, this is a pretty inefficient code. I am looking for a way to substantially speed up the computation, possibly avoiding the Transpose functions.

I am dealing with a multiplication of two- and three- dimensional arrays. Since the arrays are large, I am compiling a function to deal with it. The code of the function, with a working example, is as follows:

(*array dimensions*)
pp = 30; nn = 3000; 

(*sample data*)
a = Table[1., {p, 1, pp}, {n1, 1, nn}, {n2, 1, nn}];
b = Table[1., {p, 1, pp}, {n1, 1, nn}];
c = Table[1., {p, 1, pp}, {n1, 1, nn}];
d = Table[1., {p, 1, pp}, {n1, 1, nn}];

(*defining compiled function*)
With[{P = pp}, (*localizing P to avoid MainEvaluate*)
   y=Compile[{{a, _Real, 3}, {b, _Real, 2}, {c, _Real, 2}, {d, _Real, 2}},
        Module[{num, den, i1, out},
             i1 = b c d ;
             num = a i1;
             den = Table[Total[num[[p]]], {p, 1, P}]; 
             out = Table[ 
                 Transpose[Transpose[num[[p]]]/den[[p]]], 
               {p, 1, P}];  
             out
        ] 
     ]
]; 
  • a: pp x nn x nn
  • b,c,d: pp x nn

To be clear, the purpose of the last instruction in the compiled function is to:

  • take "page p" of num (I am thinking of a 3-dimensional array as a book, with pages, and tables NxN in each page);
  • divide each column by the correspondent row of den, to obtain again an NxN matrix (this is why I am using a double Transpose instruction).
  • create pp pages of such matrices.

In my application, pp~30 and nn~3000. It takes about 9-10 seconds to run this function on my (fairly fast) computer:

 DateString[]
 test = y[a, b, c, d];
 DateString[]

 Out[363]= "Thu 22 Jun 2017 09:10:43"

 Out[365]= "Thu 22 Jun 2017 09:10:54"

Since I will need to run it thousands of times, this is a pretty inefficient code. I am looking for a way to substantially speed up the computation, possibly avoiding the Transpose functions.

added 100 characters in body
Source Link
Fred
  • 347
  • 1
  • 7

I am dealing with a multiplication of two- and three- dimensional arrays. Since the arrays are large, I am compiling a function to deal with it. The code of the function is as follows:

y=Compile[{{a, _Real, 3}, {b, _Real, 2}, {c, _Real, 2}, {d, _Real, 2}},
     Module[{num, den, i1, out},
          i1 = b c d ;
          num = a i1;
          den = Table[Total[num[[p]]], {p, 1, P}]; 
          out = Table[ 
              Transpose[Transpose[num[[p]]]/den[[p]]], 
            {p, 1, P}];  
          out
     ] 
 ]; 

All inputs are real and positive numbers. The dimensions of the input variables are as follows:

  • a: P x N x N
  • b,c,d: P x N

The variable P is also compiled within the module, and there are no MainEvaluate in the code.

To be clear, the purpose of the last instruction is to:

  • take "page p" of num (I am thinking of a 3-dimensional array as a book, with pages, and tables NxN in each page);
  • divide each column by the correspondent row of den, to obtain again an NxN matrix (this is why I am using a double Transpose instruction).
  • create P pages of such matrices.

In my application, P~30 and N~3000. It takes about 10 seconds to run this function. Since I will need to run it thousands of times, this is a pretty inefficient code. I am looking for a way to substantially speed up the computation, possibly avoiding the Transpose functions.

I have found that the most expensive part it the last instruction, that seems to take around 4 seconds by itself. A couple of seconds go before the first instruction and after the last computation. Three-four seconds are taken by the first three instructions.

Any suggestions are greatly appreciated.

I am dealing with a multiplication of two- and three- dimensional arrays. Since the arrays are large, I am compiling a function to deal with it. The code of the function is as follows:

y=Compile[{{a, _Real, 3}, {b, _Real, 2}, {c, _Real, 2}, {d, _Real, 2}},
     Module[{num, den, i1, out},
          i1 = b c d ;
          num = a i1;
          den = Table[Total[num[[p]]], {p, 1, P}]; 
          out = Table[ 
              Transpose[Transpose[num[[p]]]/den[[p]]], 
            {p, 1, P}];  
          out
     ] 
 ]; 

All inputs are real and positive numbers. The dimensions of the input variables are as follows:

  • a: P x N x N
  • b,c,d: P x N

To be clear, the purpose of the last instruction is to:

  • take "page p" of num (I am thinking of a 3-dimensional array as a book, with pages, and tables NxN in each page);
  • divide each column by the correspondent row of den, to obtain again an NxN matrix (this is why I am using a double Transpose instruction).
  • create P pages of such matrices.

In my application, P~30 and N~3000. It takes about 10 seconds to run this function. Since I will need to run it thousands of times, this is a pretty inefficient code. I am looking for a way to substantially speed up the computation, possibly avoiding the Transpose functions.

I have found that the most expensive part it the last instruction, that seems to take around 4 seconds by itself. A couple of seconds go before the first instruction and after the last computation. Three-four seconds are taken by the first three instructions.

Any suggestions are greatly appreciated.

I am dealing with a multiplication of two- and three- dimensional arrays. Since the arrays are large, I am compiling a function to deal with it. The code of the function is as follows:

y=Compile[{{a, _Real, 3}, {b, _Real, 2}, {c, _Real, 2}, {d, _Real, 2}},
     Module[{num, den, i1, out},
          i1 = b c d ;
          num = a i1;
          den = Table[Total[num[[p]]], {p, 1, P}]; 
          out = Table[ 
              Transpose[Transpose[num[[p]]]/den[[p]]], 
            {p, 1, P}];  
          out
     ] 
 ]; 

All inputs are real and positive numbers. The dimensions of the input variables are as follows:

  • a: P x N x N
  • b,c,d: P x N

The variable P is also compiled within the module, and there are no MainEvaluate in the code.

To be clear, the purpose of the last instruction is to:

  • take "page p" of num (I am thinking of a 3-dimensional array as a book, with pages, and tables NxN in each page);
  • divide each column by the correspondent row of den, to obtain again an NxN matrix (this is why I am using a double Transpose instruction).
  • create P pages of such matrices.

In my application, P~30 and N~3000. It takes about 10 seconds to run this function. Since I will need to run it thousands of times, this is a pretty inefficient code. I am looking for a way to substantially speed up the computation, possibly avoiding the Transpose functions.

I have found that the most expensive part it the last instruction, that seems to take around 4 seconds by itself. A couple of seconds go before the first instruction and after the last computation. Three-four seconds are taken by the first three instructions.

Any suggestions are greatly appreciated.

Source Link
Fred
  • 347
  • 1
  • 7

Fast manipulation of large dimensional arrays

I am dealing with a multiplication of two- and three- dimensional arrays. Since the arrays are large, I am compiling a function to deal with it. The code of the function is as follows:

y=Compile[{{a, _Real, 3}, {b, _Real, 2}, {c, _Real, 2}, {d, _Real, 2}},
     Module[{num, den, i1, out},
          i1 = b c d ;
          num = a i1;
          den = Table[Total[num[[p]]], {p, 1, P}]; 
          out = Table[ 
              Transpose[Transpose[num[[p]]]/den[[p]]], 
            {p, 1, P}];  
          out
     ] 
 ]; 

All inputs are real and positive numbers. The dimensions of the input variables are as follows:

  • a: P x N x N
  • b,c,d: P x N

To be clear, the purpose of the last instruction is to:

  • take "page p" of num (I am thinking of a 3-dimensional array as a book, with pages, and tables NxN in each page);
  • divide each column by the correspondent row of den, to obtain again an NxN matrix (this is why I am using a double Transpose instruction).
  • create P pages of such matrices.

In my application, P~30 and N~3000. It takes about 10 seconds to run this function. Since I will need to run it thousands of times, this is a pretty inefficient code. I am looking for a way to substantially speed up the computation, possibly avoiding the Transpose functions.

I have found that the most expensive part it the last instruction, that seems to take around 4 seconds by itself. A couple of seconds go before the first instruction and after the last computation. Three-four seconds are taken by the first three instructions.

Any suggestions are greatly appreciated.