# Manually converting TensorFlow models to Mathematica

Is

Mathematica (v12, my third, semi-informed, attempt at manual conversion)

c = {n, s, p} \[Function]
ConvolutionLayer[n, s, "PaddingSize" -> p(*,"Biases"\[Rule]n*)];
p = PoolingLayer[{2, 2}, {2, 2}];
n = NetChain[<|(*3*)
"c1" -> {"x" -> c[032, 5, 2], "h" -> Ramp, "p" -> p},
"c2" -> {"x" -> c[064, 5, 2], "h" -> Ramp, "p" -> p},
"c3" -> {"x" -> c[128, 3, 1], "h" -> Ramp, "p" -> p},
"fc" -> {
"h_pool_flat" -> FlattenLayer[],
"x" -> LinearLayer,
"h_fc1" -> Ramp,
"h_fc1_drop" -> DropoutLayer[],
"y" -> LinearLayer[h // Length]
},
"sm" -> SoftmaxLayer[]
|>,
"Input" ->
NetEncoder[{"Image", {64, 64}, ColorSpace -> "Grayscale"}],
"Output" -> NetDecoder[{"Class", h}]
]


a good conversion of

Tensorflow (v1, source network from an IBM demo project) # First convolutional layer. 32 feature maps.
W_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable()
x_conv1 = tf.nn.conv2d(x_image, W_conv1, strides=[1, 1, 1, 1],
padding='SAME')
h_conv1 = tf.nn.relu(x_conv1 + b_conv1)
h_pool1 = tf.nn.max_pool(h_conv1, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')

# Second convolutional layer. 64 feature maps.
W_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable()
x_conv2 = tf.nn.conv2d(h_pool1, W_conv2, strides=[1, 1, 1, 1],
padding='SAME')
h_conv2 = tf.nn.relu(x_conv2 + b_conv2)
h_pool2 = tf.nn.max_pool(h_conv2, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')

# Third convolutional layer. 128 feature maps.
W_conv3 = weight_variable([3, 3, 64, 128])
b_conv3 = bias_variable()
x_conv3 = tf.nn.conv2d(h_pool2, W_conv3, strides=[1, 1, 1, 1],
padding='SAME')
h_conv3 = tf.nn.relu(x_conv3 + b_conv3)
h_pool3 = tf.nn.max_pool(h_conv3, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')

# Fully connected layer. Here we choose to have 1024 neurons in this layer.
h_pool_flat = tf.reshape(h_pool3, [-1, 8*8*128])
W_fc1 = weight_variable([8*8*128, 1024])
b_fc1 = bias_variable()
h_fc1 = tf.nn.relu(tf.matmul(h_pool_flat, W_fc1) + b_fc1)

# Dropout layer. This helps fight overfitting.
keep_prob = tf.placeholder(tf.float32, name=keep_prob_node_name)
h_fc1_drop = tf.nn.dropout(h_fc1, rate=1-keep_prob)

# Classification layer.
W_fc2 = weight_variable([1024, num_classes])
b_fc2 = bias_variable([num_classes])
y = tf.matmul(h_fc1_drop, W_fc2) + b_fc2

# This isn't used for training, but for when using the saved model.
tf.nn.softmax(y, name=output_node_name)

1. How can the Mathematica model be improved to match the Tensorflow version exactly?
2. Is there a resource anywhere to learn the correspondences between the two?
3. Specifically, I am not sure about
1. tf.matmul == ???
4. Why does uncommenting (*, "Biases" -> n*) cause convergence to either take extremely long or perhaps completely fail (I have not waited)?
5. Why are the dimensions in different order between the two?
1. I remember seeing in an answer here that Mathematica was planning a change of orders due to a recent MXNet update - I am not sure which versions are involved / whether this has already happened.
• Some attemtions: github.com/GalAster/WLNet-ModelZoo/…, same padding = (new_height – 1) × S + F - W – GalAster Dec 10 '19 at 11:53
• @GalAster - do these not export just weights and biases!?, I mean what the model has learned, but not the network chains or graphs themselves? – Cetin Sert Dec 10 '19 at 12:07
• Take a closer look, this project contains the complete conversion process – GalAster Dec 10 '19 at 12:59