8
$\begingroup$

I'm surprised how slow ConvolutionLayer works with TargetDevice->"CPU". For instance this code:

data = RandomReal[{-.1, .1}, {1, 1024, 1024}];
net = NetInitialize@
   NetChain[{ConvolutionLayer[1, 3], Ramp}, 
    "Input" -> Dimensions[data]];
net[data, TargetDevice -> "CPU"]; // RepeatedTiming

calculates a simple convolution of a 1024x1024 buffer with a 3x3 filter. This takes 36ms on my PC, using 8 cores. This seems very slow.

For comparison: If I use low-level IPP functions to calculate a convolution of a 1024x1024 buffer (64bit floating point data type) with a 3x3 filter, it takes about 1.2ms, using only a single core.

Is the mxnet CPU convolution implementation really 250x less efficient than IPP?

Clarification: The IPP (Intel Performance Primitives) is a low-level CPU library for signal-processing and image processing tasks, optimized for Intel CPUs. So I'm comparing two different CPU implementations.


Add:

A completely naive C implementation (below) takes about 3ms for a 1024x1024 * 3x3 convolution. Isn't 'ConvolutionLayer[1,3]' doing more or less the same? Why does it take 10 times as long using 8 times as many cores?

for (int y=1;y<h-1; ++y)
    for (int x = 1; x < w-1; ++x)
    {
        double s = 0;
        for (int dy = -1; dy <= 1; ++dy)
            for (int dx = -1; dx <= 1; ++dx)
                s += src[(y + dy)*w + x + dx] * flt[dx + 1 + 3 * (dy + 1)];
        dst[y*w + x] = s;
    }
$\endgroup$
  • 3
    $\begingroup$ From what you say in your question, it seems the answer is "yes" 😉 $\endgroup$ – Carl Lange May 7 at 16:32
  • 1
    $\begingroup$ Yep. The GPU market is booming right now, especially with "external GPUs" precisely for applications like this. You'll find a lot on the topic (and the reasons why GPUs excel in these scenarios) in a quick Google search. $\endgroup$ – user6014 May 7 at 17:11
  • 5
    $\begingroup$ @user6014: I'm not sure I follow. What does this have to do with GPUs? IPP is a CPU-only library. It seems as if the mxnet CPU implementation is simply inefficient as hell, and could be >200 times faster on the CPU if it was optimized well. $\endgroup$ – Niki Estner May 7 at 19:14
1
$\begingroup$

Exploration of the problem. This is not an answer.

SeedRandom[0];
img = RandomImage[1, {1024, 1024}, ColorSpace -> "Grayscale"];

enc = NetEncoder[{"Image", {1024, 1024}, ColorSpace -> "Grayscale"}];

eimg = enc@img;

conv = ConvolutionLayer[1, 3, "Input" -> {1, 1024, 1024}] // NetInitialize;

conv@eimg; // RepeatedTiming

{0.0428, Null}

w = NetExtract[conv, "Weights"][[1, 1]] // Normal;

ImageConvolve[img, Reverse /@ Reverse@w, Padding -> None]; // RepeatedTiming

{0.0055, Null}

(conv@eimg)[[1, 1, ;; 10]]

{-1.27131,-1.58433,-2.09876,-1.88886,-1.69361,-2.02711,-1.13003,-0.975968,-1.394,-1.95289}

ImageConvolve[img, Reverse /@ Reverse@w, Padding -> None] // ImageData // #[[1, ;; 10]] &

{-1.27131,-1.58433,-2.09876,-1.88886,-1.69361,-2.02711,-1.13003,-0.975968,-1.394,-1.95289}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.