3
$\begingroup$

Does Mathematica 11 have access with either the Wikipedia or CommonCrawl FastText embeddings? Maybe this is built-in through the Neural Net Repository or available through a NetEncoder?

$\endgroup$
  • $\begingroup$ I'm not sure this is currently possible... But there are many other embeddings you can use in the neural net repo, at least. It does seem like the format of the pre-built models is relatively parseable so you may be able to use them that way. $\endgroup$ – Carl Lange Oct 16 '18 at 7:28
4
$\begingroup$

Download and extract the data.

f = OpenRead["wiki-news-300d-1M.vec"];

The first line of the file contains the number of words in the vocabulary and the size of the vectors.

{n, d} = Read[f, {Number, Number}]

{999994, 300}

t = Table[Read[f, Join[{Word}, ConstantArray[Real, d]]], {10(*n*)}];
tokens = t[[;; , 1]];
emb = t[[;; , 2 ;;]];

https://mathematica.stackexchange.com/a/171569

net = NetChain[
  {
   EmbeddingLayer["Weights" -> Append[emb, ConstantArray[0, d]]],
   LongShortTermMemoryLayer[10],
   SequenceLastLayer[],
   2,
   SoftmaxLayer[]
   },
  "Input" -> NetEncoder[{"Tokens", tokens}],
  "Output" -> NetDecoder[{"Class", {0, 1}}]
  ]

enter image description here

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.