How to find similar words from the provided list?

Question

Let's say I have created a list of words that are acceptable in a certain context.

list = {...};

Now given a new or some existing word from the list I want to find the words nearest in meaning to it from the list.

I tried:

Nearest[list,"chosenWord"]

But this gives the words with the least difference in characters rather than something sensible in meaning. I want all the words to get assigned a numerical distance from the chosen word but in meaning.

Moreover, almost all words carry with them a sentiment so if the numerical difference could also take into account the formality and the tone of the word that would be even better.

Can someone guide me if there is a builtin feature or some Neural Net that can do what I am trying to get at?

Answer 1

PART 2 - UPDATE: Semantics vs Physics

As an exercise of applications of the method outlined in the PART 1 - INTRO below, it is fun and useful to see the differences between semantical and physical worlds. Semantics a very relative, for example to the neural net architecture or training set. Do not expect semantic results to be necessarily logical and physical.

Lets take a look at the colors. Here is a list of some named colors:

yourWORDS={"white","yellow","blue","red","green","black","brown","azure","ivory","teal",
"silver","purple","gray","orange","maroon","aquamarine","coral","fuchsia",
"wheat","lime","crimson","khaki","pink","magenta","gold","plum","olive","cyan"};

And the ELMo run on them to get numerical space:

netNonContextual=
NetTake[
    NetModel["ELMo Contextual Word Representations Trained on 1B Word Benchmark"],
{NetPort["Input"],"embedding"}]

word2vec=With[{words=yourWORDS},AssociationThread[words->netNonContextual[words][[All,1]]]];

You can easily build a graph showing nearest neighbor of each color semantics-wise:

Graph[
    Values[word2vec],
    EdgeList[NearestNeighborGraph[Values[word2vec],1,DirectedEdges->True]],
VertexLabels->Thread[Values[word2vec]->Keys[word2vec]],
VertexLabelStyle->13,
GraphStyle->"Prototype",
GraphLayout->"SpringEmbedding"]

We can run the same procedure on real physical color values:

trueCOLOR=AssociationThread[yourWORDS->Interpreter["Color"][yourWORDS]]

And of course you will get a very different graph :-) The difference most prominently seen, probably, in the relation of Black and White colors. Black and White often go close in texts, hence their closest proximity in semantic graph. An of course physically its quite the opposite.

With[{vec=List@@@ColorConvert[Values[trueCOLOR],"LAB"]},

Graph[
    vec,
    EdgeList[NearestNeighborGraph[vec,1,DirectedEdges->True]],
VertexLabels->Thread[vec->Keys[trueCOLOR]],
VertexLabelStyle->14,
GraphStyle->"Prototype",
GraphLayout->"SpringEmbedding"]

]

PART 1 - INTRO

First you need something that converts semantics into numerical (vector, matrix, etc.) space. And a lot depends on a specific converter that you use. It is better to know exactly which converter you are using, how it works, and experiment with other similar converters. Usually people use already trained neural nets (NNS), because then you do not have to train them yourself. As @carl-lange already suggested in comments, look carefully through text-related NNs here and examples in their docs:

https://resources.wolframcloud.com/NeuralNetRepository/inputdomain/Text

Thee key here is choose something good for the goal you need. Again you probably will need to experiment with a few different NNs. Here is an example I adopted from docs for ELMo which is considered state-of-the-art.

https://resources.wolframcloud.com/NeuralNetRepository/resources/ELMo-Contextual-Word-Representations-Trained-on-1B-Word-Benchmark

Extract the non-contextual part of the net:

netNonContextual=
NetTake[
    NetModel["ELMo Contextual Word Representations Trained on 1B Word Benchmark"],
{NetPort["Input"],"embedding"}]

NOTE: you can do this with with "context" too, which would allow strings having more than a single token (word). Take your word list and precompute the context-independent embeddings -- for the word list here I use all adjectives from English dictionary given by WordList:

yourWORDS=WordList["Adjective"];
word2vec=With[{words=yourWORDS},AssociationThread[words->netNonContextual[words][[All,1]]]];

Option TargetDevice -> "GPU" in netNonContextual would speeds up things, but don't use it if you do not have GPU. Now you can find your nearest words to a specific one:

In[]:= Nearest[word2vec,word2vec["angry"],5]
Out[]= {angry,furious,outraged,annoyed,frustrated}


In[]:= Nearest[word2vec,word2vec["lovely"],5]
Out[]= {lovely,wonderful,lively,beautiful,delightful}

Be alert to potential issues. For instance, most of the modern models are "unaware" of antonyms:

In[]:= Nearest[word2vec,word2vec["unhappy"],5]
Out[]= {unhappy,happy,angry,frustrated,uncomfortable}

You see that "unhappy" is considered close to "happy", which is true in a sense of close usability of words in text, but not true in terms of antonyms or a tint of sentiment.

If you want to tinker more with sentiment look at "Sentiment" section in Sentiment Language Model:

https://resources.wolframcloud.com/NeuralNetRepository/resources/Sentiment-Language-Model-Trained-on-Amazon-Product-Review-Dataset

And finally, by studying architecture of relevant NNs, you can design your own converter, - that is what many people often do when a sophisticated task is at hand.