0
$\begingroup$

Does anyone know of a way to get the BERT model working for word embeddings in Mathematica? Doesn’t seem to be available on GitHub or the Wolfram Net Repo...

$\endgroup$
  • 5
    $\begingroup$ Do you have any links or more details you could add to your question? The paper the model is specified in, for instance, or models built in other languages? $\endgroup$ – Carl Lange Mar 15 at 8:16
  • 1
    $\begingroup$ Well I just use pytorch to do this, but in general having proper tutorial on training transformer networks in mma would be great - a key paper is “all you need is attention” great read. But Bert is on the tf hub - that’s what I use. $\endgroup$ – M.R. Apr 11 at 19:12
  • 6
    $\begingroup$ If nothing else, here's the link to the actual network mentioned, and here's the paper. Please add some more details to your post so that potential answerers don't need to do their own research before they can even start to try and help. $\endgroup$ – Carl Lange Apr 15 at 17:20
11
+100
$\begingroup$

As Wolfram Language 12 is just released so are the following new relevant models compatible with it. They were just added to the Wolfram Neural Network repository:

There are more updates in Neural Nets Repo that can be quickly scanned at:

https://resources.wolframcloud.com/NeuralNetRepository/all

There is more to come of course, stay tuned !

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.