2
$\begingroup$

I have a question regarding neural network graphs.

In R there is the following code where a user can add hidden layers with different number of nodes:

n <- neuralnet(medv ~ crim+zn+indus+chas+nox+rm+age+dis+rad+tax+ptratio+b+lstat,
           data = data,
           hidden = c(12,7),
           linear.output = F,
           lifesign = 'full',
           rep=1)

At the above table c(12,7) means that there will be 2 hidden layers with 12 nodes for the first and 7 for the second. And we plot it, it gives the following plot: Plot of a neural net

Is there anything similar for Mathematica? I have searched and I have found some info although it seems it is not working.

At How to change NeuralNetwork options? there is a code that says at the point to define the Method we have to apply

data = {Range[1, 100], RandomReal[NormalDistribution[0, 0.01], {100}] - Range[0, 0.1, 0.1/(100     - 1)]} // Transpose;    
data = Rule @@@ data;    
model = Predict[data, Method -> "NeuralNetwork"];    
PredictorInformation[model]    

And then we apply the following: Method -> {"NeuralNetwork", "L2Regularization" -> 0.12, "HiddenLayers" -> {4, 3, 3}}

In my case when I run the above code, the following message appears: After the run of the code

And the following as a result: enter image description here

Is there something like analytical notes that explain all that?

$\endgroup$

1 Answer 1

3
$\begingroup$

By default, Predict with "NeuralNetwork" as a method creates a neural network with 2 hidden layers of width 50 (nodes)

data=Rule@@@({Range[1,100],RandomReal[NormalDistribution[0,0.01],{100}]-Range[0,0.1,0.1/(100-1)]}//Transpose);
model = Predict[data, Method -> "NeuralNetwork"];
Information[model, "MethodParameters"]

Gives <|"NetworkDepth" -> 2, MaxTrainingRounds -> 300|>, which implies there are 2 hidden layers.

To extract the NetChain object in the NetGraph object created by Predict, we can use:

NetExtract[(First@model)["Model", "Network"], {"1"}]

NetChain

Which shows us that there are two hidden layers with 50 nodes each, with SELU activation functions.

We can change the amount of hidden layers with the option NetworkDepth:

model2 = Predict[data, Method -> {"NeuralNetwork", "NetworkDepth" -> 1}]

NetChain model 2

If you want to have more control over the structure of the neural network, you can use NetChain instead to define it:

NetChain model 3

Which has 3 hidden layers, of width 4, 3 and 3 respectively, and SELU activation function.

Then, you can train this model with the data:

model3trained = NetTrain[model3, data]

And then use it for predictions:

Prediction

Then you can generate plots of the network.

MXNet Style Graph

Mathematica Full Summary Graph

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.