# How to solve the xor problem with neural network

As this playground show after you click this button, just four levels can solve the xor problem. So I try to simulate it in Mathematica

## Generate test points

    disk1 = Disk[{0, 0}, 1, {0, Pi/2}];
disk2 = Disk[{0, 0}, 1, {Pi/2, Pi}];
disk3 = Disk[{0, 0}, 1, {Pi, 3 Pi/2}];
disk4 = Disk[{0, 0}, 1, {3 Pi/2, 2 Pi}];
pts = RandomPoint[#, RandomInteger[{30, 40}]] & /@ {disk1, disk2,
disk3, disk4};
Graphics[{PointSize[0.02], {Red, Point[Catenate[pts[[{1, 3}]]]], Blue,
Point[Catenate[pts[[{2, 4}]]]]}}]


## Training the network

data = Flatten[
Thread /@ {Catenate[pts[[{1, 3}]]] -> 1,
Catenate[pts[[{2, 4}]]] -> 0}];
net = NetChain[{4, Ramp, 2, Ramp,
SoftmaxLayer["Output" -> NetDecoder[{"Class", {0, 1}}]]},
"Input" -> 2];
trainednet =
NetTrain[net, data, ValidationSet -> Scaled[.2],
TargetDevice -> "GPU"]


## Show the trained result

ContourPlot[trainednet[{x, y}], {x, -1, 1}, {y, -1, 1},
Epilog -> {PointSize[0.02], Red, Point[Catenate[pts[[{1, 3}]]]],
Blue, Point[Catenate[pts[[{2, 4}]]]]}]


We get a good result as the above image, but actually,we often get a frustrated result like following

I'm confused, because the playground always converge well. Why I use same network layer but cannot get same good result? How to improve it?

• Without testing anything: The Ramp before the SoftmaxLayer looks suspicious Commented Oct 10, 2017 at 15:11
• @nikie Then. :) And as my textbook,that is not a problem.
– yode
Commented Oct 10, 2017 at 15:15
• @nikie It will be better a little indeed
– yode
Commented Oct 10, 2017 at 15:57