3
$\begingroup$

I'm trying to understand why I can decode encoded word tokens?

s = "Why isn't this decoding?";
e = NetEncoder[{"Tokens", "English", "IgnoreCase" -> True}];
d = NetDecoder@e;
encoded = e@s;
d[encoded]

enter image description here

$\endgroup$
2
$\begingroup$

Ah, I missed this in the documentation:

The input to the decoder input is either a vector of probabilities or a sequence of probability vectors. Each probability vector sums to 1. The length of each probability vector is the size of the vocabulary.

Should be:

d[UnitVector[40236, #] & /@ encoded]

enter image description here

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.