# Can someone help me to understand sequence-to-sequence transformation using (Wolfram-specific) recurrent neural nets?

I've been learning about recurrent neural networks lately and I think I'm starting to get the basic idea of how they work. I'm particularly interested in the sequence transformation capabilities of these nets for applications in both NLP and generative art. I've played with a few simple (non-recurrent) nets in Mathematica, but would like to learn more about how to implement recurrent sequence-to-sequence learning.

I've read the Wolfram tutorial Sequence Learning and NLP with Neural Networks, and I'm particularly interested in the section titled Integer Addition with Variable-Length Output. If I understand correctly, sequence-to-sequence learning involves converting a sequence to a vector, and then converting that vector into another sequence. I understand (mostly) the "sequence-to-vector" parts with things like SequenceLastLayer[]. However, I'm still not entirely clear from the tutorial how the "vector-to-sequence" part of this works. Are there other, more descriptive examples somewhere?

• I wrote the tutorial, and would definitely like to improve it to be more understandable (it is one of the more complicated neural net application examples...). Are there any parts of going from a vector to a sequence that you understand, or is the whole thing a complete mystery? May 22, 2018 at 9:12
• If I understand correctly--and I'm still not 100% sure I do--there are basically two stages to sequence-to-sequence transformation: the encoder (sequence-to-vector) stage and the decoder (vector-to-sequence) stage. The encoder stage somehow feeds its state to the decoder stage, and the decoder always tries to make sure that any given vector (along with an associated encoder state) will result in (roughly) the same output sequence. Hopefully this is something like the right idea. May 23, 2018 at 17:00