Return to Answer

Using quadruple LSTM(state size 64) - Best result of LSTM family.

Visualize the effect of auto-encoder of Bast result(using quadruple LSTM(state size 64)),the effect is pretty good.

Using octuple LSTM(state size 16):

Contrary to my expectations, it didn't get a better result compare to the above net :)

Using quadruple LSTM(state size 64) - Best result of LSTM family.

Visualize the effect of auto-encoder of Bast result(using quadruple LSTM(state size 64)),the effect of clustering is pretty good.

Using octuple LSTM(state size 16),it is very hard to train because too many parameters.

Finally,we could simulate the process the auto-encoder of DNN. Using the structure of Best result of quadruple LSTM(state size 64),but the embedding size change it to 16.

Finally,Come back to this question.

We can simulate the process the auto-encoder of DNN, not only the result is not bad but also it can learn the embedding size(16) if using the structure of Best result of quadruple LSTM(state size 64)

Visualize the effect of auto-encoder of Bast result(using quadruple LSTM(state size 64)):

Extreme Test:  Enlarge the number of LSTM-staks is not a efficency way,it will make the net hard to train. That is having more LSTM-stacks,The results are not necessarily good.

Finally,we could simulate the process the auto-encoder of DNN. Using the structure of quadruple LSTM(state size 64),but the embedding size is 16. So result is good.

Visualize the effect of auto-encoder of Bast result(using quadruple LSTM(state size 64)),the effect is pretty good.

Extreme Test: 

Enlarge the number of LSTM-staks is not an efficency way, it will make the net hard to train. In other words, Increasing the number of LSTM-stacks may not be effective.

Finally,we could simulate the process the auto-encoder of DNN. Using the structure of Best result of quadruple LSTM(state size 64),but the embedding size change it to 16.

We can see the result is not bad.