# Neural network giving different answers

I'm very new to machine learning and pretty new to Mathematica, trying to implement a simple neural network that acts as a "XOR" gate. Here's my code:

net = NetChain[{LinearLayer[2], LogisticSigmoid, LinearLayer[],
LogisticSigmoid}];
trained = NetTrain[net, {{1, 1} -> 0, {1, .8} -> 0, {0, .2} -> 0, {0,
1} -> 1, {1, 0} -> 1, {0, 0} -> 0}];
trained[{.2, .8}, None]


I'm confused because this code produces different outputs when I run it multiple times. I would have thought that Mathematica would just use some sort of least squares algorithm to compute weights based on my input, but it seems that Mathematica is doing something much more complicated. Can you explain why this is happening? And can you explain how I might eliminate this indeterminacy?

• I'm not sure how to answer this question without basically explaining how neural network training works - and there already are many good tutorials on that. In a nutshell: It's using (a form of) gradient descent from a random starting point. If you don't want that, neural networks are probably not the technology you want. Commented Nov 28, 2017 at 16:55
• There is some stochasticity to it, yes. Commented Nov 28, 2017 at 21:22
• Placing BlockRandom around the call to NetTrain makes things deterministic. Commented Nov 29, 2017 at 3:16
• @ChipHurst: in general, no. First, for GPU training, BlockRandom has no effect on cuDNN primitives at the moment. Second, there will almost always be some stochasticity due to all of our numerics being completely parallelized (things like numeric addition are not associative, meaning slightly different answers are gotten if parallel threads do things in different orders). Commented Dec 7, 2017 at 17:41
• @Sebastian I guess I left this comment because Equal @@ Table[BlockRandom[(* OP's code *)], 10] returns True on my machine. Commented Dec 7, 2017 at 22:22

1. The weights and biases of neural networks must be initialized. Setting them all to zero is a bad idea from a machine learning point of view. So, often they are initialized randomly. See the Mathematica documentation for NetInitialize for more information or disabling random initialization.