# How does NetDecoder for Tokens work?

I'm trying to understand why I can decode encoded word tokens?

s = "Why isn't this decoding?";
e = NetEncoder[{"Tokens", "English", "IgnoreCase" -> True}];
d = NetDecoder@e;
encoded = e@s;
d[encoded]


• Instead of using 40236, you may need to change the number. There should be an error message if it's wrong indicating the right number. The underlying list of words may change. – Searke Jun 14 '19 at 16:33

## 1 Answer

Ah, I missed this in the documentation:

The input to the decoder input is either a vector of probabilities or a sequence of probability vectors. Each probability vector sums to 1. The length of each probability vector is the size of the vocabulary.

Should be:

d[UnitVector[40236, #] & /@ encoded]