Complex numbers in neural networks

Question

I am trying to construct a neural network which fits a collection of functions from (complex-valued) vectors to complex numbers. After constructing a suitable network, I was surprised to see that complex values do not seem to play nicely with the standard functions. In particular, when trying to start my network training with NetTrain[], the following error message is displayed if the training data contains any complex numbers:

LibraryFunction::cfta: Argument {0. +1. I,0. +1. I,0. +1. I,0. +1. I,0. +1. I,0. +1. I,0. +1. I,0. +1. I} at position 1 should be a rank -1 tensor of machine-size real numbers.

Is there a way to use complex numbers with Mathematica's machine-learning functions? I cannot find any references to such accommodations in the documentation.

Answer 1

As v12.2, mathematica does not support raw complex number to be used in training model but like other unsupported formats, converting them to an acepeted formats like number, vector, etc will help you to achieve your goal. It's very common because there are so many types that can't directly processed but by converting them, we could see the power of Machine learning almost anywhere.

We can't use 2+2I in training set but we could use a vector {2,2} in which the first element relate to real part and second to imaginary, and then applying some function like Complex to that vector will give us 2+2I.

It means when we have a set that contains unsupported formats, you should apply a function called Encoder to translate them in a supported format, let the computer process on that, give you the result in the specified supported format and again apply another function called Decoder to get you the result in the type you entered the data.

Example

rawData = {1+2 I -> 3+5 I, 5+7 I -> 1+6 I};
(* Complex -> Complex *)

(*--Encoding--*)
convertedData = ReIm[#[[1]]] -> ReIm@#[[2]] & /@ rawData;
(*Out: {{1, 2} -> {3, 5}, {5, 7} -> {1, 6}} *)

trained = NetTrain[LinearLayer[], convertedData]

now use the trained model to evaluate 1+3 I:

trained[{1, 3}]
(*Out: {4.76502, 5.57501} *)

(*--Decoding--*)
Apply[Complex, trained[{1, 3}]]
(*Out: 4.76502 + 5.57501 I *)

Mathematica 11.3 and above

Mathematica 11.0 introduced built-in NetEncoder and NetDecoder with different types, which we will use Function type which introduced in 11.3 to specify our own Encoder/Decoder. For some reason, probably my limited knowledge I couldn't use full power of decoder, so I will convert only the output of training set:

rawData = {1+2 I -> 3+5 I, 5+7 I -> 1+6 I};

convertedData2 = #[[1]] -> ReIm@#[[2]] & /@ rawData;
(*Out: {1 + 2 I -> {3, 5}, 5 + 7 I -> {1, 6}} *)

encoder = NetEncoder[{"Function", ReIm[#] &, {2}}];
decoder = NetDecoder[{"Function", Complex[#[[1]], #[[2]]] &}];

net2 = NetChain[{LinearLayer[2, "Input" -> encoder, "Output" -> decoder]}]
trained2 = NetTrain[net2, convertedData2]

After training, there is no need to convert Input or Output:

trained2[1 + 3 I]
(*Out: 4.76502 + 5.57501 I *)

All the code tested in Mathematica 12.2.