# How to find multi-word synonyms?

I’m looking to write a semantic distance function for phrases (5 words or less) that can map multi-word synonyms together. My domain is e-commerce fashion - here are some required examples for a solution:

1. “Red shoes” would be close to “Orange sandals”

2. “Blue leopard purse” would be close to “violet spotted handbag”

Here's the type of data that I'm starting with, a static list of synonyms:

exampleSyns = CloudGet @ "https://www.wolframcloud.com/obj/832467b7-87d8-4544-a0ba-3b461a9a4e99";


Approaches I've tried

I have tried the obvious thing of using the distance of BERT vectors, but this doesn’t work well enough at the moment.

Are lists of synonyms and antonyms built-in somewhere? If so we could use them, but this seems like a general enough problem that there are probably easier approaches that I don’t know of.

I’m sure there’s an easier approach to handling this problem in Mathematica that people in NLP could help with.

Related & References

• Can you share your BERT code? That would have been my first guess as well. – Carl Lange Aug 28 '19 at 6:04
• NetPairEmbeddingOperator[NetChain[{EmbeddingLayer[],LongShortTermMemoryLayer[],SequenceLastLayer[]}] – Alexey Golyshev Aug 28 '19 at 6:26
• Using a large enough text corpus can provide these kind of synonyms. Do you want to deal with large text data, or you just want to utilize some database or a trained neural network? – Anton Antonov Oct 17 '19 at 10:35
• @M.R. I plan to post an answer after two-three days. – Anton Antonov Oct 25 '19 at 6:00
• @AntonAntonov two-three or twenty three? ;) – M.R. Nov 6 '19 at 2:04

(Extended comment, not an answer. From the comments to it by OP I do no see the point expanding it further into full blown answer. Thanks to everyone who reacted to it...)

## Procedure outline

1. Ingest text data.

2. Make a LSAMon object with sentence granularity level.

3. Find topics.

• It is important to use the syntagmatic perspective.
4. Using the topics find word pairs that appear adjacent to each other in text.

5. Consider speeding up that process with:

• a proper recommender, or

• a prefix tree with frequencies for word pairs.

6. Select popular enough phrases.

7. Tokenize the text with the found 2-word phrases.

8. Make a second LSAMon object with sentence granularity level.

9. Find phrases statistical thesauri.

## Result

Here is a screenshot of Step 9 results:

Values[
Map[Pick[#, StringMatchQ[#, __ ~~ "→" ~~ __]] &, tblThesaurus]]


Here is text version of the phrase thesaurus:

(* {{"beautiful→view", "spacious→home"},
{"convenient→location", "great→value", "wonderful→time"},
{"cute→neighborhood", "accommodating→hosts"},
{"great→place", "great→stay", "perfect→location"},
{"highly→recommend", "great→location", "quiet→neighborhood", "perfect→location", "walking→distance", "great→place"},
{"host→canceled", "automated→posting"},
{"nice→place", "super→close", "beautiful→house", "excellent→location"},
{"perfect→location", "loved→staying", "quiet→neighborhood", "really→enjoyed", "great→hosts", "short→drive", "awesome→location"},
{"time→staying", "excellent→hosts", "wonderful→hosts"},
{"walking→distance", "great→stay", "short→walk", "easy→access", "fort→lauderdale", "super→clean", "short→drive"}} *)

• The bounty is doubled because it ran out, please add more complete answer if possible – M.R. Nov 26 '19 at 8:00
• What is LSAMon? Can you use BERT or any neural approach? – M.R. Nov 26 '19 at 8:02
• LSAMon is a package / software monad for dimension reduction. Its use in the described procedure can be replaced with word2vec or BERT. It is most likely I am not going to post here the complete solution, but in Community. (Within a day or two...) – Anton Antonov Nov 26 '19 at 12:07
• Can you address the multiword examples in the post please: {"Red shoes”, “Orange sandals”, “Blue leopard purse”, “violet spotted handbag”} – M.R. Nov 26 '19 at 16:44
• Monads are just design patterns right? How would you BERT here, not clear at all to me... If you could add that in here it would be helpful, then I'll read your post on w-community. – M.R. Nov 26 '19 at 16:44