# Why won't nominally compilable function compile? (err Compile::cpintlt)

Context: I have millisecond timestamped sample data that I want to convert to a uniformly sampled series. This I do by padding each interval with repeated values (pbp ~ 'pad by position'}.

The following function works as intended, (I would credit the trick of using a range to generate an explicit list of indices for speed if I could remember who it came from...)

SetAttributes[pbp, HoldAll];
pbp[s_,idx_]:=
Module[{ca,len, idx2},
idx2 = idx - idx[[1]] +1;
ca = ConstantArray[0,len=Last[idx]-First[idx]+1];
(ca[[#1[[1]]]]=#1[[2]]) & /@ Table[{Range[idx2[[i]],idx2[[i+1]]-1], s[[i]]}, {i,1,Length[idx2]-1}];
ca[[len]]=Last[s];
Return[ca];
];


For nSamples = 100k pbp runs in about 1.5s but my data set is orders of magnitude larger and I have other things to do afterwards so I would like any performance gain I can get, but I can't get it to compile. Being aware of some ConstantArray compilation issues I substituted a table, and as far as I can tell from CompilerFunctions[] there is nothing controversial in this:

pbpCompiled=
Compile[{{s, _Real, 1}, {idx, _Integer, 1}},
Module[{ca,len, idx2},
idx2 = idx - idx[[1]] +1;
len=Last[idx]-First[idx]+1;
ca = Table[0, {j,1,len}];
(Part[ca, Part[#, 1]] = Part[#, 2]) & /@ Table[{Range[idx2[[i]],idx2[[i+1]]-1], s[[i]]}, {i,1,Length[idx2]-1}];
ca[[len]]=Last[s];
ca
],
CompilationTarget -> "C", "RuntimeOptions"->"Speed"];


(NB, it was more idiomatically written but became less so as I tried to tease elements apart for debugging.)

However, I get this error (3x before further cpintIt are suppressed)

CompileGetElement[SystemPrivateCompileSymbol[0],SystemPrivateCompileSymbol[1]][[1]] at position 2 of ca[[CompileGetElement[SystemPrivateCompileSymbol[0],SystemPrivateCompileSymbol[1]][[1]]]] should be either a nonzero integer or a vector of nonzero integers; evaluation will use the uncompiled function

Q1 - what is wrong with the definition of the function to be compiled/how can it be fixed?

Q2 - is there some other uncompiled approach to the problem that would be significantly faster than the uncompiled version above?

The following can be used to provide some test data...

nSamples = 100000;
sampleSeries = RandomReal[{0, 1}, nSamples];
sampleTimes = IntegerPart@(1000 Accumulate[RandomReal[{0.001, 0.500}, nSamples]]);


then called as e.g. pbp[sampleSeries, sampleTimes]

• What are you going to do with the output of pbp? Why not just create a zeroth order InterpolatingFunction from the data? Mar 27, 2019 at 19:40
• @CarlWoll. I'm going to moving average it with a 1s window, then subtract the m.a. and do stats on residuals and more processing of the m.a. Didn't follow your suggestion but tried it anyway on trust with ifun = Interpolation[Transpose[{sampleSeries, sampleTimes}], InterpolationOrder -> 0]; Plot[ifun[x], {x, 614, 2383}] (small segment at the beginning of 100 samples spanning 493 to ). Got a flat line and error; seems ifun[x] for any x -> "Input value {nnn} lies outside the range of data in the interpolating function". Maybe this is a Q2 answer but I can't tell :( Mar 27, 2019 at 20:05
• p[s_, idx_] := Join @@ MapThread[ ConstantArray[#1, #2] &, {s, Append[Differences[idx], 1]}]; a bit more concise, s/b quicker...
– ciao
Mar 27, 2019 at 21:38
• @ciao I'll have to mentally unpack and then try that that when I get home; thanks. Mar 28, 2019 at 7:21
• Have you tried using TimeSeries with its built-in TimeSeriesResample method? reference.wolfram.com/language/guide/TimeSeries.html Mar 28, 2019 at 11:02

An array in Compile must have elements of a consistent type. The Table[] in the OP has, on the face of it, a mixture of integers and reals; however, the integers are promoted to type Real. Consequently, their values are not suitable as arguments to Part. Here is one workaround to prove the point, but one might consider alternatives:

pbpCompiled =
Compile[{{s, _Real, 1}, {idx, _Integer, 1}},
Module[{ca, len, idx2}, idx2 = idx - idx[[1]] + 1;
len = Last[idx] - First[idx] + 1;
ca = Table[0, {j, 1, len}];
(Part[ca, Round@Part[#, 1]] = Part[#, 2]) & /@   (* Note the use of Round[] *)
Table[{Range[idx2[[i]], idx2[[i + 1]] - 1], s[[i]]}, {i, 1,
Length[idx2] - 1}];
ca[[len]] = Last[s];
ca], CompilationTarget -> "C", "RuntimeOptions" -> "Speed"]


Update: There is a further problem that I overlooked. The entry in the Table command is not a tensor, bu has the form {_List, _Real}:

{Range[idx2[[i]], idx2[[i + 1]] - 1], s[[i]]}


Beyond that, ca is initialized to a tensor of integers and later assigned real values, so it needs to be initialized to an array of real zeros, 0. Also Part[ca, Range[..]] led to a MainEvaluate, so I made it an explicit loop.

pbpCompiled = Compile[{{s, _Real, 1}, {idx, _Integer, 1}},
Module[{ca, len, idx2},
idx2 = idx - idx[[1]] + 1;
len = Last[idx] - First[idx] + 1;
ca = Table[0., {j, 1, len}];
Do[
Do[Part[ca, j] = CompileGetElement[s, i], {j, idx2[[i]],
idx2[[i + 1]] - 1}], {i, 1, Length[idx2] - 1}];
ca[[len]] = Last[s];
ca], CompilationTarget -> "C", "RuntimeOptions" -> "Speed"]


If we make a similar changes in the uncompiled pbp, then the arrays stay packed and pbp[] runs just a little slower than the compiled version.

pbp[s_, idx_] := Module[{ca, len, idx2},
idx2 = idx - idx[[1]] + 1;
ca = ConstantArray[0., len = Last[idx] - First[idx] + 1];
Do[ca[[Range[idx2[[i]], idx2[[i + 1]] - 1]]] = s[[i]],
{i, 1, Length[idx2] - 1}];
ca[[len]] = Last[s];
Return[ca];];

• Thank you; I didn't realise the Ints in that Table would get promoted, because they were in a separate sub-list. Now it compiles, proving your point, but at runtime it complains {{Ints}, Real} (see [1] below) "should be a rank 1 tensor of machine sized numbers", which is at least different (why not caught @ compile?). [1] for a short run, {{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25, 26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48, 49,50,<<86>>},0.776588} Mar 28, 2019 at 5:20
• @JulianMoore It was late and I didn't have time to think past what was the compiler error. I've updated my answer with improvements to both pbp and pbpCompiled. Mar 28, 2019 at 12:16
• using nSamples = 1000000 you pbp was faster than pbpcompiled and took only 0.083s, which I find quite astonishing. How might I have "known" that Do was a better way to approach this; performance optimisation seems a very dark art. [NB I also spotted that the ca tensor was initialised to int 0 not real 0, but the real problem was elsewhere] Where can one learn about such things, and CompileGetElement etc.? Or is it just time, effort and MMA SE Q&A? Mar 28, 2019 at 21:01