Return to Revisions

I compare many net structure using LSTM. Then finding multi-stacks LSTM will get a more accurate result.

Note:All the networks use this code as basic model,batch size 256,epochs 10

net = NetGraph[{encoder, decoder, MeanSquaredLossLayer[]}, 
               {1 -> 2 -> NetPort["Output"], 2 -> NetPort[3, "Input"], NetPort["Input"] -> NetPort[3, "Target"]}, 
               "Input" -> NetEncoder[{"Image", {28, 28}, "Grayscale", "MeanImage" -> meanImage}], 
               "Output" -> NetDecoder[{"Image", "Grayscale"}]]

Baseline network:

encoder = NetChain[{FlattenLayer[], 100, Ramp, 50, Ramp, 16}];
decoder = NetChain[{50, Ramp, 100, Ramp, 784, ReshapeLayer[{1, 28, 28}]}];

Using single LSTM(state size 16):

encoder = NetChain[{ReshapeLayer[{28, 28}], LongShortTermMemoryLayer[16], 
                    SequenceLastLayer[]}];
decoder = NetChain[{ReplicateLayer[28], LongShortTermMemoryLayer[28], 
                    ReshapeLayer[{1, 28, 28}]}];

Using single LSTM(state size 64):

encoder = NetChain[{ReshapeLayer[{28, 28}], LongShortTermMemoryLayer[64], 
                    SequenceLastLayer[]}];
decoder = NetChain[{ReplicateLayer[28], LongShortTermMemoryLayer[28], 
                    ReshapeLayer[{1, 28, 28}]}];

Using double LSTM(state size 16):

encoder = NetChain[{ReshapeLayer[{28, 28}], LongShortTermMemoryLayer[16], 
                    LongShortTermMemoryLayer[16], SequenceLastLayer[]}];
decoder = NetChain[{ReplicateLayer[28], LongShortTermMemoryLayer[28], 
                    LongShortTermMemoryLayer[28], ReshapeLayer[{1, 28, 28}]}];

Using double LSTM(state size 64):

encoder = NetChain[{ReshapeLayer[{28, 28}], LongShortTermMemoryLayer[64], 
    LongShortTermMemoryLayer[64], SequenceLastLayer[]}];
decoder = NetChain[{ReplicateLayer[28], LongShortTermMemoryLayer[28], 
    LongShortTermMemoryLayer[28], ReshapeLayer[{1, 28, 28}]}];

Using triple LSTM(state size 64):

encoder = NetChain[{ReshapeLayer[{28, 28}], 
                    LongShortTermMemoryLayer[64], 
                    LongShortTermMemoryLayer[64], 
                    LongShortTermMemoryLayer[64], SequenceLastLayer[]}];
decoder = NetChain[{ReplicateLayer[28], 
                    LongShortTermMemoryLayer[28], 
                    LongShortTermMemoryLayer[28], 
                    LongShortTermMemoryLayer[28], ReshapeLayer[{1, 28, 28}]}];

Using triple LSTM(state size 64) and DNN:

encoder = NetChain[{ReshapeLayer[{28, 28}], 
                    LongShortTermMemoryLayer[64], 
                    LongShortTermMemoryLayer[64], 
                    LongShortTermMemoryLayer[64], 
                    SequenceLastLayer[], 32, ElementwiseLayer["SELU"], 16}];
decoder = NetChain[{32, ElementwiseLayer["SELU"], 64, ReplicateLayer[28], 
                    LongShortTermMemoryLayer[28], 
                    LongShortTermMemoryLayer[28], 
                    LongShortTermMemoryLayer[28], ReshapeLayer[{1, 28, 28}]}];

USing quadruple LSTM(state size 64) - Best:

encoder = NetChain[{ReshapeLayer[{28, 28}], 
                    LongShortTermMemoryLayer[64], 
                    LongShortTermMemoryLayer[64], 
                    LongShortTermMemoryLayer[64], 
                    LongShortTermMemoryLayer[64], SequenceLastLayer[]}];
decoder = NetChain[{ReplicateLayer[28], 
                    LongShortTermMemoryLayer[28], 
                    LongShortTermMemoryLayer[28], 
                    LongShortTermMemoryLayer[28], 
                    LongShortTermMemoryLayer[28], ReshapeLayer[{1, 28, 28}]}];

Visualize the effect of auto-encoder of Bast result(using quadruple LSTM(64)):