I am trying to use the VGG16 architecture available in the Wolfram Neural Network Repository to classify the Fashion MNIST dataset. The VGG 16 by default accepts images of size 3x224x224. However the images in the Fashion MNIST are grayscale of dimensions 1x28x28. I have tried to tune the model by using the command

newNet = NetReplacePart(vgg16, "Input"->NetEncoder[{"Image", {28, 28}, ColorSpace->"Grayscale"}]]

But this throws an error message that says

NetEncoder producing a 1x28x28 array of real numbers cannot be attached to a port Input which must be 3x224x224 array of real numbers.

This is highly frustrating and I cannot find a way through. Any help will be deeply appreciated. Thanks in advance.

  • $\begingroup$ Crossposted here. $\endgroup$ Commented Oct 19, 2021 at 0:14

1 Answer 1


It's not immediately obvious to me that transfer learning is appropriate here, considering the datasets are rather different in dimensions - but here goes since you asked:

  1. You have to train (at-least) an input convolution layer to take your input images to the size the first trained convolution layer expects , here that's (64,224,224)
inputNet = NetChain[{
   ConvolutionLayer[64, {3, 3}, "PaddingSize" -> {1, 1}], Ramp},
  "Input" -> 
   NetEncoder[{"Image", {224, 224}, ColorSpace -> "Grayscale"}]]
  1. That means we also have to discard the first trained conv layer from VGG-16
vgg = NetModel["VGG-16 Trained on ImageNet Competition Data"];
tempNet = Take[vgg, {3, -4}]
  1. We can then attach some linear layers at the end together with a Decoder to predict our classes. I was too lazy to download the MNIST fashion data, so here's some dummy data for 10 classes
trainSet = Table[
   RandomImage[1, {224, 224}] -> f[RandomInteger[{1, 10}]], 100];

newNet = NetChain[<|"input" -> inputNet, "pretrainedNet" -> tempNet, 
   "linearNew" -> LinearLayer[10], "softmax" -> SoftmaxLayer[]|>, 
  "Output" -> NetDecoder[{"Class", Array[f, 10]}]]
  1. Finally, we can re-train only the input convolutional net and the linear net
trainedNet = 
 NetTrain[newNet, trainSet, 
  LearningRateMultipliers -> {"input" -> 1, "linearNew" -> 1, _ -> 0}]
  • $\begingroup$ thanks a lot for your detailed answer. It certainly helped me gain a deeper understanding of the process to be followed. However, I still can't figure out how to work with the original images in the FASHION_MNIST dataset which are of size 28x28 and not 224x224. Also the original images are grauscale and not in RGB. As per your solution I believe the net is expecting images of size 224x224. As a sidelight in Python Keras I can do the same thing by specifying the input_size parameter while initializing the VGG net. But I cannot seem to do the same here. $\endgroup$
    – A. Mustafi
    Commented Oct 19, 2021 at 3:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.