1
$\begingroup$

I am trying to use Predict and Classify functions. I am interested what NetworkDepth really means when I do like this

Classify[{1, 2, 3, 4} -> {1, 1, 2, 2}, 
Method -> {"NeuralNetwork", "NetworkDepth" -> 2}]

in this example 2 is the same as they tell here like Human Brain has 6 layers? In other words, if I want to make predictor/classifier similar that human has in the brain I have to put NetworkDepth -> 6. Or NetworkDepth is completely different and has nothing in common with layers in Human Cerebral cortex?

$\endgroup$
  • 3
    $\begingroup$ I think those are 6 layers of tissue not 6 layers of neurons. I'm 99% sure the numbers mean something completely different and have nothing in common. The human brain isn't a multilayer backpropagation network. $\endgroup$ – Niki Estner Oct 23 '17 at 13:04
7
$\begingroup$

There is only an analogy between an actual brain and a so called "neural network" which were inspired from those. The way the information is processed in each is different. You can look here for more detail information wikipedia article "Artificial Neural Network"...

The simplest artificial neural networks, can be viewed as a stack of successive mathematical transformations with tunable (learnable) parameters, each transformation result being the input of the next one. In those the "Depth" notion usually refers to the size of those operations, each of them also being called a "layer" by analogy.

This "Depth" notion is behind the expression "deep learning" often used nowadays. But for more complex networks it does not generalize into a well defined quantity (for instance, when complex transformations, or when networks branches are involved...).

In Classify/Predict "Neural Network" method, some typical network architecture have been implemented ("FullyConnected", "Convolutional", "Recurrent") each being base on a characteristic "module" (in the "Convolutional" case, this module is a ConvolutionLayer followed by a PoolingLayer, an ElementwiseLayer, and a BatchNormalizationLayer). "Depth", in this context, is the number of those module which are chained together in the neural networks generated by Classify/Predict (which can incorporate others layers).

You can play with all options summarized on the "Neural Network" method page to see which hyper-parameter best serve your purpose... Keep in mind that Classify/Predict does this automatically by default (Automatic). As a rule of thumb let Classify/Predict choose the appropriate value or use "Depth" value around 2 (2 or 3) which will probably give better results.

Eventually, you can access all the functionalities of the underlaying artificial neural networks framework of Mathematica: see the Neural Network Guide in the documentation and the rich examples in the Examples>Applications section of the NetTrain reference page.

$\endgroup$
  • $\begingroup$ Thanks. Very good answer! $\endgroup$ – Zlelik Oct 31 '17 at 21:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.