10
$\begingroup$

I'd like to get the top 50, the bottom 50 and an ordered list of all Wolfram Language symbols based on their "Ranks".
With

allWLS = EntityList["WolframLanguageSymbol"];

this returns the top 50

Pick[allWLS, 
 UnitStep[Replace[EntityValue["WolframLanguageSymbol", "Ranks"], 
     m_Missing -> {{Infinity, Infinity}}, {1}][[All, 1, 2]] - 51], 0]

this should return the bottom 50

maxRank = Max[Replace[EntityValue["WolframLanguageSymbol", "Ranks"], 
 m_Missing -> {{0, 0}}, {1}][[All, 1, 2]]]

Pick[allWLS, 
 UnitStep[Replace[EntityValue["WolframLanguageSymbol", "Ranks"], 
     m_Missing -> {{0, 0}}, {1}][[All, 1, 2]] - (maxRank - 51)], 1]

but returns the bottom 38 (all having an "All" rank of 4469), and

orderedWLS = 
  allWLS[[Ordering[
     Replace[EntityValue["WolframLanguageSymbol", "Ranks"], 
       m_Missing -> {{Infinity, Infinity}}, {1}][[All, 1, 2]]]]];

returns a list of all Wolfram Language symbols ordered by their "All" rank.
The bottom 50 can then be found with

orderedWLS[[-50 - # ;; -# - 1]] &@
 Count[EntityValue["WolframLanguageSymbol", "Ranks"], _Missing]

My questions are:

  1. Is there a way to get this done faster within the Entity framework? Especially using Replace feels kind of odd here. There should be a way to get for example the top 50 by using Interval[{1, 50}] directly for the Entitys, similar to the following example for "VersionIntroduced":

    WolframLanguageData[
     EntityClass[
      "WolframLanguageSymbol", {"VersionIntroduced" -> Interval[{10.2, 10.3}]}]]
    
  2. Is there a way to do this faster by circumventing the use of Entitys completely or partially (e.g. by preprocessing the downloaded data)?

$\endgroup$
  • 1
    $\begingroup$ The problem is that WolframLanguageData[] itself uses Entity[] to return results, and the ranks seem to only be accessible through Alpha or the corresponding EntityValue[]. $\endgroup$ – J. M. will be back soon Oct 23 '15 at 5:15
  • $\begingroup$ @J.M. I was thinking of using something like WolframLanguageData[ EntityClass[ "WolframLanguageSymbol", {"VersionIntroduced" -> Interval[{10.2, 10.3}]}]], but couldn't make it work for "Ranks". $\endgroup$ – Karsten 7. Oct 23 '15 at 5:29
  • $\begingroup$ @J.M. To me the real problem (with respect to 1.) seems to be that "Ranks" are lists and not single values. $\endgroup$ – Karsten 7. Oct 23 '15 at 5:42
  • 1
    $\begingroup$ I think that's because the ranks aren't for one field, but for several; e.g. WolframLanguageData["Sin", "Ranks"]. My comment was more intended to address point 2; that is; I don't see any obvious way to avoid going online just to get the function ranks. $\endgroup$ – J. M. will be back soon Oct 23 '15 at 5:45
  • $\begingroup$ @J.M. An example where multiple fields are no problem WolframLanguageData["Plot", "FunctionalityAreas"], WolframLanguageData[ EntityClass[ "WolframLanguageSymbol", {"FunctionalityArea", "PlottingFunctions"}]]. $\endgroup$ – Karsten 7. Oct 23 '15 at 7:03
11
$\begingroup$

This gives 5 symbols with the highest rank in "All":

In[1]:= EntityValue["WolframLanguageSymbol", "Ranks", 
  "EntityAssociation"] // Query[TakeSmallest[5] /* Keys, "All"]

Out[1]= {Entity["WolframLanguageSymbol", "List"], 
 Entity["WolframLanguageSymbol", "Rule"], 
 Entity["WolframLanguageSymbol", "Times"], 
 Entity["WolframLanguageSymbol", "Power"], 
 Entity["WolframLanguageSymbol", "Plus"]}

It is currently not possible to get the same result without retrieving all the data because a query like

EntityList[EntityClass[type, "property" -> value]]

is executed only when value is a simple expression like a number, quantity, entity, ..., or one of a selection of operators like ContainsAny[{entity1, ...}], GreaterThan[x], ... .

$\endgroup$
4
$\begingroup$

The "Corpus" qualifier can be used to generate an efficient query.

EntityList[
 EntityClass["WolframLanguageSymbol", 
  EntityProperty["WolframLanguageSymbol", "Ranks", {"Corpus" -> "All"}] -> 
   TakeSmallest[50]]]

Out

There are examples for this in the documentation here and here now.

$\endgroup$
  • 1
    $\begingroup$ It's worth noting that this seems to be part of an update of the Entity framework and not of Mathematica itself. Therefore this also works in version 10 and not only in version 11. $\endgroup$ – Karsten 7. Aug 14 '16 at 15:36
  • $\begingroup$ OTOH, one now wonders if it was already built-in and was just not documented before… $\endgroup$ – J. M. will be back soon Aug 14 '16 at 15:51
3
$\begingroup$

One possibility is to convert the result of EntityValue[] into a Dataset[] and then perform ranking/sorting queries. Here is what I came up with:

wlranks = Dataset[Association /@ MapAt["Name" -> # &, Prepend[#2, #1] & @@@
                  DeleteMissing[EntityValue["WolframLanguageSymbol",
                                            {"Name", "Ranks"}], 1, 1], {All, 1}]]

Here's how to query the top 50 functions by their "All" ranking:

wlranks[TakeSmallestBy[#All &, 50], "Name"] // Normal
   {"List", "Rule", "Times", "Power", "Plus", "Set", "Alternatives", "Null",
    "Blank", "NoWhitespace", "Pattern", "$Failed", "CompoundExpression", "Slot",
    "Part", "Sqrt", "RGBColor", "None", "Pi", "SetDelayed", "Function", "Equal",
    "Subscript", "Automatic", "True", "Directive", "I", "Map", "Opacity",
    "RuleDelayed", "FinancialData", "GrayLevel", "Sin", "False", "If", "Hold",
    "Quantity", "ReplaceAll", "CityData", "Line", "Cos", "Condition", "Less",
    "Style", "And", "E", "Table", "HoldComplete", "Word", "Length"}

A problem I noticed with querying the bottom 50 is that there seems to be a lot of ties at the bottom. With that caveat, you can use TakeLargestBy[] to extract the bottom 50. A sorted list of the functions ranked by "All" is returned by wlranks[SortBy[#All &], "Name"] // Normal. Similar operations can be done for the other ranks, e.g. "StackExchange":

wdd[TakeSmallestBy[#StackExchange &, 50], "Name"] // Normal
   {"List", "Times", "Set", "Power", "Rule", "Blank", "Pattern", "Slot",
    "CompoundExpression", "Plus", "Part", "Function", "SetDelayed", "Map",
    "Equal", "Null", "Pi", "ReplaceAll", "Table", "Apply", "Sin", "All", "True",
    "Length", "Sqrt", "RuleDelayed", "Range", "Cos", "If", "Derivative", "False",
    "First", "Less", "Flatten", "Greater", "Module", "Plot", "Transpose",
    "PlotRange", "I", "None", "Red", "LessEqual", "And", "ImageSize",
    "StringCases", "With", "Graphics", "PatternTest", "Dynamic"}
$\endgroup$
  • 1
    $\begingroup$ What does this mean, the "rank" of a symbol? Just how frequently it appears in a given codebase? $\endgroup$ – Oleksandr R. Oct 25 '15 at 2:58
  • $\begingroup$ @Oleksandr, yes, that was my reading of it; taking the last one as an example, List is the most popular Mathematica function in StackExchange posts, followed by Times. $\endgroup$ – J. M. will be back soon Oct 25 '15 at 3:02
  • $\begingroup$ @OleksandrR. EntityValue["WolframLanguageSymbol", "Ranks", "Description"] returns "ranks of usage". One can get the "frequencies of usage" by using "Frequencies" instead of "Ranks". $\endgroup$ – Karsten 7. Oct 25 '15 at 8:16
  • 1
    $\begingroup$ One can avoid making two downloads and save some time by using EntityValue["WolframLanguageSymbol", {"Name", "Ranks"}] instead of WolframLanguageData[WolframLanguageData[], {"Name", "Ranks"}]. $\endgroup$ – Karsten 7. Oct 25 '15 at 8:18
  • $\begingroup$ @Karsten, I've changed it. Thanks! $\endgroup$ – J. M. will be back soon Oct 25 '15 at 12:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.