10
$\begingroup$

I'd like to get the top 50, the bottom 50 and an ordered list of all Wolfram Language symbols based on their "Ranks".
With

allWLS = EntityList["WolframLanguageSymbol"];

this returns the top 50

Pick[allWLS, 
 UnitStep[Replace[EntityValue["WolframLanguageSymbol", "Ranks"], 
     m_Missing -> {{Infinity, Infinity}}, {1}][[All, 1, 2]] - 51], 0]

this should return the bottom 50

maxRank = Max[Replace[EntityValue["WolframLanguageSymbol", "Ranks"], 
 m_Missing -> {{0, 0}}, {1}][[All, 1, 2]]]

Pick[allWLS, 
 UnitStep[Replace[EntityValue["WolframLanguageSymbol", "Ranks"], 
     m_Missing -> {{0, 0}}, {1}][[All, 1, 2]] - (maxRank - 51)], 1]

but returns the bottom 38 (all having an "All" rank of 4469), and 

orderedWLS = 
  allWLS[[Ordering[
     Replace[EntityValue["WolframLanguageSymbol", "Ranks"], 
       m_Missing -> {{Infinity, Infinity}}, {1}][[All, 1, 2]]]]];

returns a list of all Wolfram Language symbols ordered by their "All" rank.
The bottom 50 can then be found with

orderedWLS[[-50 - # ;; -# - 1]] &@
 Count[EntityValue["WolframLanguageSymbol", "Ranks"], _Missing]

My questions are:

  1. Is there a way to get this done faster within the Entity framework? Especially using Replace feels kind of odd here. There should be a way to get for example the top 50 by using Interval[{1, 50}] directly for the Entitys, similar to the following example for "VersionIntroduced":

    WolframLanguageData[
 EntityClass[
  "WolframLanguageSymbol", {"VersionIntroduced" -> Interval[{10.2, 10.3}]}]]

  2. Is there a way to do this faster by circumventing the use of Entitys completely or partially (e.g. by preprocessing the downloaded data)?

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ The problem is that WolframLanguageData[] itself uses Entity[] to return results, and the ranks seem to only be accessible through Alpha or the corresponding EntityValue[]. $\endgroup$
    – J. M.'s missing motivation ♦
    Commented Oct 23, 2015 at 5:15
  • $\begingroup$ @J.M. I was thinking of using something like WolframLanguageData[ EntityClass[ "WolframLanguageSymbol", {"VersionIntroduced" -> Interval[{10.2, 10.3}]}]], but couldn't make it work for "Ranks". $\endgroup$
    – Karsten7
    Commented Oct 23, 2015 at 5:29
  • $\begingroup$ @J.M. To me the real problem (with respect to 1.) seems to be that "Ranks" are lists and not single values. $\endgroup$
    – Karsten7
    Commented Oct 23, 2015 at 5:42
  • 1
    $\begingroup$ I think that's because the ranks aren't for one field, but for several; e.g. WolframLanguageData["Sin", "Ranks"]. My comment was more intended to address point 2; that is; I don't see any obvious way to avoid going online just to get the function ranks. $\endgroup$
    – J. M.'s missing motivation ♦
    Commented Oct 23, 2015 at 5:45
  • $\begingroup$ @J.M. An example where multiple fields are no problem WolframLanguageData["Plot", "FunctionalityAreas"], WolframLanguageData[ EntityClass[ "WolframLanguageSymbol", {"FunctionalityArea", "PlottingFunctions"}]]. $\endgroup$
    – Karsten7
    Commented Oct 23, 2015 at 7:03

3 Answers 3

Reset to default
11
$\begingroup$

This gives 5 symbols with the highest rank in "All":

In[1]:= EntityValue["WolframLanguageSymbol", "Ranks", 
  "EntityAssociation"] // Query[TakeSmallest[5] /* Keys, "All"]

Out[1]= {Entity["WolframLanguageSymbol", "List"], 
 Entity["WolframLanguageSymbol", "Rule"], 
 Entity["WolframLanguageSymbol", "Times"], 
 Entity["WolframLanguageSymbol", "Power"], 
 Entity["WolframLanguageSymbol", "Plus"]}

It is currently not possible to get the same result without retrieving all the data because a query like

EntityList[EntityClass[type, "property" -> value]]

is executed only when value is a simple expression like a number, quantity, entity, ..., or one of a selection of operators like ContainsAny[{entity1, ...}], GreaterThan[x], ... .

Improve this answer
$\endgroup$
5
$\begingroup$

The "Corpus" qualifier can be used to generate an efficient query.

EntityList[
 EntityClass["WolframLanguageSymbol", 
  EntityProperty["WolframLanguageSymbol", "Ranks", {"Corpus" -> "All"}] -> 
   TakeSmallest[50]]]

Out

There are examples for this in the documentation here and here now.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ It's worth noting that this seems to be part of an update of the Entity framework and not of Mathematica itself. Therefore this also works in version 10 and not only in version 11. $\endgroup$
    – Karsten7
    Commented Aug 14, 2016 at 15:36
  • $\begingroup$ OTOH, one now wonders if it was already built-in and was just not documented before… $\endgroup$
    – J. M.'s missing motivation ♦
    Commented Aug 14, 2016 at 15:51
3
$\begingroup$

One possibility is to convert the result of EntityValue[] into a Dataset[] and then perform ranking/sorting queries. Here is what I came up with:

wlranks = Dataset[Association /@ MapAt["Name" -> # &, Prepend[#2, #1] & @@@
                  DeleteMissing[EntityValue["WolframLanguageSymbol",
                                            {"Name", "Ranks"}], 1, 1], {All, 1}]]

Here's how to query the top 50 functions by their "All" ranking:

wlranks[TakeSmallestBy[#All &, 50], "Name"] // Normal
   {"List", "Rule", "Times", "Power", "Plus", "Set", "Alternatives", "Null",
    "Blank", "NoWhitespace", "Pattern", "$Failed", "CompoundExpression", "Slot",
    "Part", "Sqrt", "RGBColor", "None", "Pi", "SetDelayed", "Function", "Equal",
    "Subscript", "Automatic", "True", "Directive", "I", "Map", "Opacity",
    "RuleDelayed", "FinancialData", "GrayLevel", "Sin", "False", "If", "Hold",
    "Quantity", "ReplaceAll", "CityData", "Line", "Cos", "Condition", "Less",
    "Style", "And", "E", "Table", "HoldComplete", "Word", "Length"}

A problem I noticed with querying the bottom 50 is that there seems to be a lot of ties at the bottom. With that caveat, you can use TakeLargestBy[] to extract the bottom 50. A sorted list of the functions ranked by "All" is returned by wlranks[SortBy[#All &], "Name"] // Normal. Similar operations can be done for the other ranks, e.g. "StackExchange":

wdd[TakeSmallestBy[#StackExchange &, 50], "Name"] // Normal
   {"List", "Times", "Set", "Power", "Rule", "Blank", "Pattern", "Slot",
    "CompoundExpression", "Plus", "Part", "Function", "SetDelayed", "Map",
    "Equal", "Null", "Pi", "ReplaceAll", "Table", "Apply", "Sin", "All", "True",
    "Length", "Sqrt", "RuleDelayed", "Range", "Cos", "If", "Derivative", "False",
    "First", "Less", "Flatten", "Greater", "Module", "Plot", "Transpose",
    "PlotRange", "I", "None", "Red", "LessEqual", "And", "ImageSize",
    "StringCases", "With", "Graphics", "PatternTest", "Dynamic"}
Improve this answer
$\endgroup$
7
  • 1
    $\begingroup$ What does this mean, the "rank" of a symbol? Just how frequently it appears in a given codebase? $\endgroup$
    – Oleksandr R.
    Commented Oct 25, 2015 at 2:58
  • $\begingroup$ @Oleksandr, yes, that was my reading of it; taking the last one as an example, List is the most popular Mathematica function in StackExchange posts, followed by Times. $\endgroup$
    – J. M.'s missing motivation ♦
    Commented Oct 25, 2015 at 3:02
  • $\begingroup$ @OleksandrR. EntityValue["WolframLanguageSymbol", "Ranks", "Description"] returns "ranks of usage". One can get the "frequencies of usage" by using "Frequencies" instead of "Ranks". $\endgroup$
    – Karsten7
    Commented Oct 25, 2015 at 8:16
  • 1
    $\begingroup$ One can avoid making two downloads and save some time by using EntityValue["WolframLanguageSymbol", {"Name", "Ranks"}] instead of WolframLanguageData[WolframLanguageData[], {"Name", "Ranks"}]. $\endgroup$
    – Karsten7
    Commented Oct 25, 2015 at 8:18
  • $\begingroup$ @Karsten, I've changed it. Thanks! $\endgroup$
    – J. M.'s missing motivation ♦
    Commented Oct 25, 2015 at 12:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.