WLNet format incompatible between version 11.0 and 11.1

Question

It seems that there are some behavior changes of neural network layers from version 11.0 to 11.1 that leads to the incompatibility of WLNet format.

Consider this example:

Network exported in 11.0

net = NetInitialize@NetChain[{
    PoolingLayer[{2, 2}, {2, 2}]
    }, "Input" -> {1, 5, 5}
   ]
Export["poolinglayer.wlnet", net]

cannot be loaded back in version 11.1

Import["poolinglayer.wlnet"]
(* $Failed *)

It seems that the reason behinds this is that the default padding option has been changed from "same" in version 11.0 to "valid" in version 11.1.

So now I have a "WLNet" file that is exported in 11.0, how can I load it back in 11.1?

Also, are there other changes in the default behaviors of neural network layers?

Answer 1

Certain nets cannot be upgraded automatically from 11.0 to 11.1, but it is an explicit aim of ours to preserve upgradability whenever it is reasonably possible.

In this case, the actual convention used by MXNet bifurcated, and we choose to use one convention. Internally, both conventions are supported via a hidden parameter, but this will never be documented or supported officially. Something like this probably won't happen again, for what that's worth, MXNet as a whole is a lot more mature than it was in 11.0.

For this particular case, please note that you should be able to 'nudge' previous networks that use a PoolingLayer into a form that can be upgraded by tweaking their input or output size so that the difference between the two conventions goes away.

It is also possible to use the new PaddingLayer to manually add zero or constant padding before the PoolingLayer if it is part of a bigger network that cannot be changed. Unfortunately you'll have to import the network in chunks 'around' the PoolingLayer, after exporting the chunks from 11.0, and then add the PoolingLayer and PaddingLayer yourself.

The last thing you can do is to manually use the other, hidden convention. You can do this as follows in 11.1:

DownValues[NeuralNetworks`Private`Upgrade`updatePooling] = DownValues[NeuralNetworks`Private`Upgrade`updatePooling] /. "valid" -> "full";
Import["poolinglayer.wlnet"]

I don't like this kind of thing, because it is confusing to anyone who examines the network and of course can't see the value of the hidden convention field.

The worst kind of change that we anticipate in future is when a default of a particular layer changes and running construction code might not produce the same network as before, but upgrading a previously exported net will still produce a net that has the same behavior.

Something slightly less disruptive will almost surely happen with image-oriented layers in the next few versions. Theey will go from a CHW convention to an HWC convention. Old nets will continue to function via a "Convention" parameter, and most construction code will keep working because the default value of the convention parameter will change globally and therefore the constructed net as a whole will keep working the same way.