# Adding an interpolated function to neural networks through ElementwiseLayer

1.Trying to add an interpolated function to neural networks.

ifun = Interpolation[Table[{x, Tanh[x]}, {x, -100, 100, 0.2}],
InterpolationOrder -> 1]
Plot[{ifun[x], Tanh[x]}, {x, -4, 4}]

2.We implement the function.

net = NetChain[{30, ElementwiseLayer[ifun], 20,
ElementwiseLayer[ifun], 3, SoftmaxLayer[]}, "Input" -> {2},
"Output" -> NetDecoder[{"Class", {Red, Green, Blue}}]]

3.Error

ElementwiseLayer::invscf: InterpolatingFunction[{{-100.,100.}},... could not be symbolically evaluated as a unary scalar function.

4.Please tell me how to fix it.

Using @CarlWoll's InterpolationToPiecewise will work. ElementWiseLayer complains that the InterpolatingFunction does not "symbolically evaluate." I take that to mean that it accepts only certain expressions as symbolic expressions.

(* https://mathematica.stackexchange.com/a/212753 *)
InterpolationToPiecewise[if_, x_] :=
Module[{main, default, grid}, grid = if["Grid"];
Piecewise[{if@"GetPolynomial"[#, x - #], x < First@#} & /@
grid[[2 ;; -2]], if@"GetPolynomial"[#, x - #] &@grid[[-1]]]] /;
if["InterpolationMethod"] == "Hermite";

pwfun[x_] = InterpolationToPiecewise[ifun, x];

net = NetChain[{30, ElementwiseLayer[pwfun], 20,
ElementwiseLayer[pwfun], 3, SoftmaxLayer[]}, "Input" -> {2},
"Output" -> NetDecoder[{"Class", {Red, Green, Blue}}]]
• Thanks, it helped.Another error occurred while training the network. NetTrain::interr2 : An unknown internal error occurred.Consult Internal $\ LastInternalFailure for potential information.  Consult Internal`$ LastInternalFailure InvalidJson Non - JSON value : {{KeyAbsent, $Failed}, {KeyAbsent,$Failed}}.
– Глеб
May 15, 2020 at 15:02
• @Глеб I'm not sure what causes that. Maybe ask another question about that issue. Or ask Wolfram Support. (This is somewhat similar but unanswered: mathematica.stackexchange.com/questions/212986/…) May 15, 2020 at 16:11

Currently, piecewise function support in ElementwiseLayer is somewhat limited. It doesn't allow logical disjunctions (e.g. x<10. || x<9. ...) which were generated when PiecewiseFunction automatically unified all of the -1s and 1s in the function value list. Simplifying the expression with PiecewiseExpand would help, but then again, this would generate unsupported conditions of the form a < x <= b. Seems that ElementwiseLayer currently only supports simple inequalities inside of PiecewiseFunction.

The quickest solution here (in terms of my work) is to use the Which construct instead of PiecewiseFunction:

InterpolationToWhich[if_, x_] :=
Module[{grid}, grid = if["Grid"];
Which @@ Flatten[
{
{x < First@#, if@"GetPolynomial"[#, x - #]} & /@ grid[[2 ;; -2]],
True,
if@"GetPolynomial"[#, x - #] &@grid[[-1]]
}
]
] /; if["InterpolationMethod"] == "Hermite";

wfun[x_] = InterpolationToWhich[ifun, x];
ElementwiseLayer[wfun]

There's much to be optimized, though. One could use a binary tree of Ifs to reduce the layer's time complexity from linear to logarithmic.

• Tank you very much.You can helped me.I am funny.
– Глеб
May 19, 2020 at 6:39