8
$\begingroup$

Say I have a comma-separated list of words. For example sake, let it be things I have seen during my trip to a safari zone. It would look something like that:

zebra, lion, car, ranger, teacup, honey badger, lake,

and so on. I would like Mathematica to automatically separate it in different categories such as:

"animals", "objects", "humans", "scenery"

Is there a way to do this in Mathematica 10, possibly using the WolframAlpha functionalities?

$\endgroup$
  • $\begingroup$ Do you know how to define lists ? How to manipulate them (basic operations) ? Have you tried anything ? $\endgroup$ – Sektor Jun 20 '15 at 17:30
  • $\begingroup$ Fair comment. I do know how to manipulate lists, but I am not quite sure about the machine learning capabilities of Mathematica, and all of the stuff using the curated database. $\endgroup$ – Andrea Jun 20 '15 at 17:32
  • 3
    $\begingroup$ SemanticInterpretation[#, AmbiguityFunction -> All] & /@ {"zebra", "lion", "car", "ranger", "teacup", "honey badger", "lake"} the results aren't very promising. $\endgroup$ – C. E. Jun 20 '15 at 17:45
  • 1
    $\begingroup$ You should look at Classify (the built-in classifier Classify["FacebookTopic", list] has too few categories unfortunately). You may also find WordData[word, "BroaderTerms"] useful. $\endgroup$ – Simon Woods Jun 20 '15 at 19:46
  • 1
    $\begingroup$ Another option - use Rlink and the RTextTools package. $\endgroup$ – Gordon Coale Jun 26 '15 at 8:10
5
$\begingroup$

Note

This answer was meant for a bit of fun but instead does a good job of managing expectations for Entity in the WL at present.

Image Identify and Google Image Search

Wolfram have heavily promoted their ImageIdentify project (http://blog.wolfram.com/2015/05/13/wolfram-language-artificial-intelligence-the-image-identification-project/) which might be a good way to categorise your terms. But we need images first.

@Szabolcs provides a method for obtaining Google Image Search results in their answer here https://mathematica.stackexchange.com/a/20485/1952

googImageSearch[term_] := 
 With[{json = Import["https://ajax.googleapis.com/ajax/services/search/images?v=1.0&q=" <> StringReplace[term, " " -> "%20"], "JSON"]}, 
  Import["url" /. ("results" /. ("responseData" /. json))[[1]]]]

Unfortunately, this API is deprecated and occasionally returns html instead of an image, which was terrifying when returned in the middle of a list of images:

StringTake[googImageSearch["lion"], 45]
(*"All you need to know about your night dreams."*)

In lieu of a better option (I don't want a Custom Google Search which is limited to 100 hits a day), I'll restrict googImageSearch to a specific domain (wikipedia):

googImageSearch[term_, site_] := 
 With[{json = 
    Import["https://ajax.googleapis.com/ajax/services/search/images?v=\
1.0&q=site:" <> site <> "+" <> StringReplace[term, " " -> "%20"], 
     "JSON"]}, 
  Import["url" /. ("results" /. ("responseData" /. json))[[1]]]];
words = {"zebra", "lion", "car", "park ranger", "teacup", 
   "badger honey", "lake"};
wikiImages = Map[googImageSearch[#, "en.wikipedia.org"] &, words];

These images are now ready for classification by ImageIdentify, I convert the Entity returned to its FullForm representation:

identifyImages = ImageIdentify /@ wikiImages;
Multicolumn[
 MapThread[
  Labeled[#1, #2, Top] &, {wikiImages, 
   FullForm /@ identifyImages}], 4, Alignment -> Center]

enter image description here

These Entity["Concept",_] have a number of "Properties" that might be useful to us:

FullForm /@ identifyImages[[1]]["Properties"]
{EntityProperty["Concept","AlternateNames"],EntityProperty["Concept","BroaderConcepts"],EntityProperty["Concept","Definition"],EntityProperty["Concept","EquivalentEntity"],EntityProperty["Concept","Name"],EntityProperty["Concept","NarrowerConcepts"],EntityProperty["Concept","SimilarEntities"],EntityProperty["Concept","SubsetConcepts"],EntityProperty["Concept","SupersetConcepts"],EntityProperty["Concept","WordDataSenses"],EntityProperty["Concept","WordNetID"]}

The Entity["Concept","EquivalentEntity"] property is the most promising, but alas provides less use than @Picket's suggested use of SemanticInterpretation:

Multicolumn[
 MapThread[
  Labeled[#1, #2, Top] &, {wikiImages, 
   FullForm /@ 
    Map[#[EntityProperty["Concept", "EquivalentEntity"]] &, 
     identifyImages]}], 4, Alignment -> Center]

enter image description here

SemanticInterpretation

@Pickett mentioned in the comments that SemanticInterpretation provides disappointing results, which it does but they can (sometimes) be improved by restricting the Entity classes within which you look.

EntityValue provides all Entity classes available, excluding "Word" is a logical choice and can be done as follows:

allEntities = EntityValue[]; 
allEntities$nowords = DeleteCases[allEntities, "Word"];
semanticInterp = (SemanticInterpretation[#1,Alternatives @@ (Entity[#1, __] & ) /@ allEntities$nowords] &) /@ words; 
semanticInterp = FullForm /@ semanticInterp

enter image description here

We can now group these words by the class of Entity returned using GroupBy:

GroupBy[Transpose[{words, semanticInterp /. Entity[a_, _] :> a}], Last]
(*<|Species->{{zebra,Species},{lion,Species},{car,Species},{honey badger,Species}},
City->{{ranger,City}},$Failed->{{teacup,$Failed}},Surname->{{lake,Surname}}|>*)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.