Does Mathematica 11 have access with either the Wikipedia or CommonCrawl FastText embeddings? Maybe this is built-in through the Neural Net Repository or available through a NetEncoder?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ I'm not sure this is currently possible... But there are many other embeddings you can use in the neural net repo, at least. It does seem like the format of the pre-built models is relatively parseable so you may be able to use them that way. $\endgroup$– Carl LangeCommented Oct 16, 2018 at 7:28
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Download and extract the data.
f = OpenRead["wiki-news-300d-1M.vec"];
The first line of the file contains the number of words in the vocabulary and the size of the vectors.
{n, d} = Read[f, {Number, Number}]
{999994, 300}
t = Table[Read[f, Join[{Word}, ConstantArray[Real, d]]], {10(*n*)}];
tokens = t[[;; , 1]];
emb = t[[;; , 2 ;;]];
https://mathematica.stackexchange.com/a/171569
net = NetChain[
{
EmbeddingLayer["Weights" -> Append[emb, ConstantArray[0, d]]],
LongShortTermMemoryLayer[10],
SequenceLastLayer[],
2,
SoftmaxLayer[]
},
"Input" -> NetEncoder[{"Tokens", tokens}],
"Output" -> NetDecoder[{"Class", {0, 1}}]
]
answered Oct 17, 2018 at 10:28