3
$\begingroup$

I'm looking to train or use pre-trained models in order to re-phrase text. Are there any natural language models in the Wolfram Neural Repository (or elsewhere) that can be used to paraphrase text?

Most text rephrasing models are not conditional, e.g. what they typically do is translate your text to another language and then back, and this will vary some of the words and phrases. But it’s deterministic so there’s no random-seed to provide or any good way to control the amount of variation.

enter image description here

I'm looking for any links/leads or examples for paraphrasing in Mathematica.

References:

$\endgroup$
  • $\begingroup$ Thanks for the bounty @M.R. $\endgroup$ – user5601 May 13 at 2:17
  • 1
    $\begingroup$ Np, this is a cool question and I'd like to see someone attack it with NetTrain :) $\endgroup$ – M.R. May 13 at 2:20
3
$\begingroup$

Update - some thoughts on Neural Nets

You can think of using bidirectional, language model and text generation networks somehow altering their architecture. For instance, BERT's underlying idea is to to fill a blank in context like:

"It is raining ____ and dogs today"

so it would yield "cats" or maybe "cows" if it is a bit amok :-) So you can randomly remove words form your sentence and run it through NN to fill in the gap with a similar term. You can also consider some NN architecture able to train on dataset of the type string-> string. You would use deterministic language translation to build a training dataset of the type

{.., "senttence1"->"paraphrase11", "senttence1"->"paraphrase12"...,"senttence2"->"paraphrase21", "senttence2"->"paraphrase22",...}

And then train a NN on it which will not be deterministic. But this requires advanced playing with NN architecture. NNs to consider:

Non-neural net

Here are simple toy-model idea, non-neural. Real application might need more heuristics and tuning, but might turn out better or simpler than a neural net. Synonyms can go pretty wild and jump word classes say from noun to verb etc. A simple trick is to find in original sentence a word class and then narrow down your synonyms accordingly. WordData can do this:

WordData[{"crazy", "Noun"}, "Synonyms", "List"]
Out[]= {"looney", "loony", "nutcase", "weirdo"}

compare:

WordData[{"crazy", "Adjective"}, "Synonyms", "List"]
Out[]= {"brainsick","demented","disturbed","dotty","gaga","half-baked","mad","screwball","sick","softheaded","unbalanced","unhinged","wild"}

You can define a randomized function:

paraphrase[sentence_,type_]:=
StringReplace[sentence,
#->RandomChoice[{#}~Join~WordData[{#,type},"Synonyms","List"]]&/@
TextCases[sentence,type]]

so given an original sentence:

sent="previously, tea had been used primarily for Buddhist monks to stay awake during meditation.";

you can sequentially replace various word cllasses"

Table[Fold[paraphrase,sent,{"Adjective","Adverb","Noun"}],5]//Column

enter image description here

This is very far from ideal. Note that you needd to add original word in case the synonyms list comes up empty. This is also based on WordNet which is a bit weird synonym-wise. You might use other means like

EntityValue[Entity["Word", "mad"], "SynonymsList"]

Out[]= {"huffy","sore","brainsick","crazy","demented","disturbed","sick","unbalanced","delirious","excited","frantic","unrestrained","harebrained","insane"}

or various available external APIs. But these might be harder to narrow down for a word class. Also note word capitalization is tricky. For WordData you need lower case mostly except proper nouns, so beginning of sentences is tricky.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks Vitaliy! Can you supply example code for the neural approach? $\endgroup$ – M.R. Apr 23 at 20:30
  • $\begingroup$ @M.R. It is too big of a project for my time currently, if i ever develop anything relevant i will png back. I was just sharing blueprint ideas. $\endgroup$ – Vitaliy Kaurov Apr 23 at 20:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.