6
$\begingroup$

Is

Mathematica (v12, my third, semi-informed, attempt at manual conversion)

c = {n, s, p} \[Function] 
   ConvolutionLayer[n, s, "PaddingSize" -> p(*,"Biases"\[Rule]n*)];
p = PoolingLayer[{2, 2}, {2, 2}];
n = NetChain[<|(*3*)
    "c1" -> {"x" -> c[032, 5, 2], "h" -> Ramp, "p" -> p},
    "c2" -> {"x" -> c[064, 5, 2], "h" -> Ramp, "p" -> p},
    "c3" -> {"x" -> c[128, 3, 1], "h" -> Ramp, "p" -> p},
    "fc" -> {
      "h_pool_flat" -> FlattenLayer[],
      "x" -> LinearLayer[1024],
      "h_fc1" -> Ramp,
      "h_fc1_drop" -> DropoutLayer[],
      "y" -> LinearLayer[h // Length]
      },
    "sm" -> SoftmaxLayer[]
    |>,
   "Input" -> 
    NetEncoder[{"Image", {64, 64}, ColorSpace -> "Grayscale"}],
   "Output" -> NetDecoder[{"Class", h}]
   ]

NetChain NetChain - flat

a good conversion of

Tensorflow (v1, source network from an IBM demo project)

tensorflow details

    # First convolutional layer. 32 feature maps.
    W_conv1 = weight_variable([5, 5, 1, 32])
    b_conv1 = bias_variable([32])
    x_conv1 = tf.nn.conv2d(x_image, W_conv1, strides=[1, 1, 1, 1],
                           padding='SAME')
    h_conv1 = tf.nn.relu(x_conv1 + b_conv1)
    h_pool1 = tf.nn.max_pool(h_conv1, ksize=[1, 2, 2, 1],
                             strides=[1, 2, 2, 1], padding='SAME')

    # Second convolutional layer. 64 feature maps.
    W_conv2 = weight_variable([5, 5, 32, 64])
    b_conv2 = bias_variable([64])
    x_conv2 = tf.nn.conv2d(h_pool1, W_conv2, strides=[1, 1, 1, 1],
                           padding='SAME')
    h_conv2 = tf.nn.relu(x_conv2 + b_conv2)
    h_pool2 = tf.nn.max_pool(h_conv2, ksize=[1, 2, 2, 1],
                             strides=[1, 2, 2, 1], padding='SAME')

    # Third convolutional layer. 128 feature maps.
    W_conv3 = weight_variable([3, 3, 64, 128])
    b_conv3 = bias_variable([128])
    x_conv3 = tf.nn.conv2d(h_pool2, W_conv3, strides=[1, 1, 1, 1],
                           padding='SAME')
    h_conv3 = tf.nn.relu(x_conv3 + b_conv3)
    h_pool3 = tf.nn.max_pool(h_conv3, ksize=[1, 2, 2, 1],
                             strides=[1, 2, 2, 1], padding='SAME')

    # Fully connected layer. Here we choose to have 1024 neurons in this layer.
    h_pool_flat = tf.reshape(h_pool3, [-1, 8*8*128])
    W_fc1 = weight_variable([8*8*128, 1024])
    b_fc1 = bias_variable([1024])
    h_fc1 = tf.nn.relu(tf.matmul(h_pool_flat, W_fc1) + b_fc1)

    # Dropout layer. This helps fight overfitting.
    keep_prob = tf.placeholder(tf.float32, name=keep_prob_node_name)
    h_fc1_drop = tf.nn.dropout(h_fc1, rate=1-keep_prob)

    # Classification layer.
    W_fc2 = weight_variable([1024, num_classes])
    b_fc2 = bias_variable([num_classes])
    y = tf.matmul(h_fc1_drop, W_fc2) + b_fc2

    # This isn't used for training, but for when using the saved model.
    tf.nn.softmax(y, name=output_node_name)
  1. How can the Mathematica model be improved to match the Tensorflow version exactly?
  2. Is there a resource anywhere to learn the correspondences between the two?
  3. Specifically, I am not sure about
    1. tf.matmul == ???
  4. Why does uncommenting (*, "Biases" -> n*) cause convergence to either take extremely long or perhaps completely fail (I have not waited)?
  5. Why are the dimensions in different order between the two?
    1. I remember seeing in an answer here that Mathematica was planning a change of orders due to a recent MXNet update - I am not sure which versions are involved / whether this has already happened.
$\endgroup$
3
  • 2
    $\begingroup$ Some attemtions: github.com/GalAster/WLNet-ModelZoo/…, same padding = (new_height – 1) × S + F - W $\endgroup$ – GalAster Dec 10 '19 at 11:53
  • $\begingroup$ @GalAster - do these not export just weights and biases!?, I mean what the model has learned, but not the network chains or graphs themselves? $\endgroup$ – Cetin Sert Dec 10 '19 at 12:07
  • 2
    $\begingroup$ Take a closer look, this project contains the complete conversion process $\endgroup$ – GalAster Dec 10 '19 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.