4
$\begingroup$

I've been learning about recurrent neural networks lately and I think I'm starting to get the basic idea of how they work. I'm particularly interested in the sequence transformation capabilities of these nets for applications in both NLP and generative art. I've played with a few simple (non-recurrent) nets in Mathematica, but would like to learn more about how to implement recurrent sequence-to-sequence learning.

I've read the Wolfram tutorial Sequence Learning and NLP with Neural Networks, and I'm particularly interested in the section titled Integer Addition with Variable-Length Output. If I understand correctly, sequence-to-sequence learning involves converting a sequence to a vector, and then converting that vector into another sequence. I understand (mostly) the "sequence-to-vector" parts with things like SequenceLastLayer[]. However, I'm still not entirely clear from the tutorial how the "vector-to-sequence" part of this works. Are there other, more descriptive examples somewhere?

Improve this question
$\endgroup$
2
  • $\begingroup$ I wrote the tutorial, and would definitely like to improve it to be more understandable (it is one of the more complicated neural net application examples...). Are there any parts of going from a vector to a sequence that you understand, or is the whole thing a complete mystery? $\endgroup$
    – Sebastian
    Commented May 22, 2018 at 9:12
  • $\begingroup$ If I understand correctly--and I'm still not 100% sure I do--there are basically two stages to sequence-to-sequence transformation: the encoder (sequence-to-vector) stage and the decoder (vector-to-sequence) stage. The encoder stage somehow feeds its state to the decoder stage, and the decoder always tries to make sure that any given vector (along with an associated encoder state) will result in (roughly) the same output sequence. Hopefully this is something like the right idea. $\endgroup$
    – Euphor
    Commented May 23, 2018 at 17:00

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.