# Why is the output of the Embedding layer so non-linear?

I have 2 nets, i.e. net1 (i.e. LinearLayer) and net2 (i.e. EmbeddingLayer) as below, while the input are just integers from 1 to 10:-

net1 = NetChain[{LinearLayer[2, "Input" -> 1]}];
net1 = BlockRandom[SeedRandom[10]; NetInitialize[net1]];
input = {#} & /@ Range[10];
output = net1[input];
ListLinePlot[output]


net2 = NetChain[{EmbeddingLayer[2, 100]}];
net2 = BlockRandom[SeedRandom[10]; NetInitialize[net2]];
input = Range[10];
output = net2[input];
ListLinePlot[output]


As you can see, net1 is linear, while net2 is obviously non-linear. But why net2 is not linear? What is the mathematics behind EmbeddingLayer? Is EmbeddingLayer also using the linear operation $y=A W + B$ or not?

Many thanks!

• My guess is that it encodes the integers with one-hot encoding and given random initial weight matrix it should behave like this. – swish Jul 28 '18 at 18:42
• Have a look in the 'Properties and Relations' section of EmbeddingLayer and LinearLayer to understand what they are computing. It should be clear then. – Sebastian Jul 29 '18 at 21:07
• Thanks, that of EmbeddingLayer make it clear. – H42 Jul 31 '18 at 4:59