# How to use FastText embeddings in a NetGraph?

Does Mathematica 11 have access with either the Wikipedia or CommonCrawl FastText embeddings? Maybe this is built-in through the Neural Net Repository or available through a NetEncoder?

• I'm not sure this is currently possible... But there are many other embeddings you can use in the neural net repo, at least. It does seem like the format of the pre-built models is relatively parseable so you may be able to use them that way. – Carl Lange Oct 16 '18 at 7:28

f = OpenRead["wiki-news-300d-1M.vec"];


The first line of the file contains the number of words in the vocabulary and the size of the vectors.

{n, d} = Read[f, {Number, Number}]


{999994, 300}

t = Table[Read[f, Join[{Word}, ConstantArray[Real, d]]], {10(*n*)}];
tokens = t[[;; , 1]];
emb = t[[;; , 2 ;;]];


https://mathematica.stackexchange.com/a/171569

net = NetChain[
{
EmbeddingLayer["Weights" -> Append[emb, ConstantArray[0, d]]],
LongShortTermMemoryLayer[10],
SequenceLastLayer[],
2,
SoftmaxLayer[]
},
"Input" -> NetEncoder[{"Tokens", tokens}],
"Output" -> NetDecoder[{"Class", {0, 1}}]
]