Does Mathematica 11 have access with either the Wikipedia or CommonCrawl FastText embeddings? Maybe this is built-in through the Neural Net Repository or available through a NetEncoder?

  • $\begingroup$ I'm not sure this is currently possible... But there are many other embeddings you can use in the neural net repo, at least. It does seem like the format of the pre-built models is relatively parseable so you may be able to use them that way. $\endgroup$
    – Carl Lange
    Oct 16, 2018 at 7:28

1 Answer 1


Download and extract the data.

f = OpenRead["wiki-news-300d-1M.vec"];

The first line of the file contains the number of words in the vocabulary and the size of the vectors.

{n, d} = Read[f, {Number, Number}]

{999994, 300}

t = Table[Read[f, Join[{Word}, ConstantArray[Real, d]]], {10(*n*)}];
tokens = t[[;; , 1]];
emb = t[[;; , 2 ;;]];


net = NetChain[
   EmbeddingLayer["Weights" -> Append[emb, ConstantArray[0, d]]],
  "Input" -> NetEncoder[{"Tokens", tokens}],
  "Output" -> NetDecoder[{"Class", {0, 1}}]

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.