17
$\begingroup$

WolframLanguageData allows us to view the most used functions across several code bases like Wolfram Alpha, the documentation, or Stack Exchange. I was curious about the results for my own collection of notebooks. I have an answer that I'll post now, but there is room for improvement.

Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
11
$\begingroup$

The following gave me satisfactory results, but there is definitely room for improvement.

It was important to delete output cells before finding input cells, because some output cells like data sets contain input cells within them that produce spurious results (lots of results from the TypeSystem context, etc). It also turned out to be important to delete graphics boxes from input cells. This led to Arrow being one of my most used functions due to a project where I was converting graph objects pasted as input into a text format used in a game. I didn't want to count this because I didn't type those graphics, even though they were in input cells. It's pretty straight forward otherwise, except remembering to drop the extra Hold you introduce to each cell when processing it.

Our main function:

listFunctions[file_] := 
 Cases[DeleteCases[Import[file], 
     Cell[_, "Output", ___] | _GraphicsBox, \[Infinity]], 
    Cell[data_, "Input", ___] :> data, \[Infinity]] // 
   Map[Drop[
      Cases[ToExpression[#, StandardForm, 
        Hold], _Symbol?(UpperCaseQ@
           StringTake[ToString@#, 1] &), \[Infinity], Heads -> True], 
      UpTo@1] &] // Catenate

Processing my directory:

counts = listFunctions /@ 
      FileNames["*.nb", 
       "c:\\users\\michael\\google drive\\notebooks", \[Infinity]] // 
     Catenate // Counts // Sort // Reverse

Some raw results:

List    42165
Slot    6781
Rule    3803
Function    3760
Part    2706
Set 2666
Null    2000
Blank   1686
CompoundExpression  1578
Map 1493
Times   1159
Pattern 1029
All 788
Plus    744
If  652
Select  616
Power   581
Import  503
Length  490
Span    460

The list number feels high, but maybe not. And it's not as easy to track down the cause of any problems there like it was with Arrow. The rest seems reasonable so I continued. My final function vocabulary size was 888, but a few weird results crept in further down the list, so I'll say 800. Although that's just my writing vocabulary. My reading vocabulary is probably a bit higher, although I consult the "dictionary" more when writing code than writing English.

A grand finale of word clouds:

WordCloud[counts]
WordCloud[counts, ScalingFunctions -> Log]
WordCloud[counts, 
 ColorNegate@
  Rasterize[
   Style["LIST", Bold, 24, FontFamily -> "Impact", FontTracking -> 2],
    ImageSize -> 80], ScalingFunctions -> Log, MaxItems -> 250, 
 WordSpacings -> 4, ImageSize -> 1600]

enter image description here enter image description here enter image description here

Improve this answer
$\endgroup$
3
  • $\begingroup$ So .m files does not count? :( I use mainly them. p.s. NotebookImport[#, "Input" -> "HeldExpression"] & may be of help $\endgroup$
    – Kuba
    Commented Sep 9, 2016 at 17:54
  • $\begingroup$ @Kuba .m definitely counts! Curious to see your results! Thanks for pointing out NotebookImport. I tried a second version of the function using it. It produces slightly fewer messages when running. The results are very comparable (the top 10 ranks are identical, most counts differing by just a percent or two), except the count for List goes up to 3 million now! listFunctions2[file_] := NotebookImport[file, "Input"] // Map[Drop[ Cases[#, _Symbol?(UpperCaseQ@ StringTake[ToString@#, 1] &), [Infinity], Heads -> True], UpTo@1] &] // Catenate[Cases[#, _List]] & $\endgroup$
    – Michael Hale
    Commented Sep 9, 2016 at 18:35
  • $\begingroup$ Yep, don't know where are those lists from. P.s. I see you didn't gave up on #s like I did :P $\endgroup$
    – Kuba
    Commented Sep 9, 2016 at 19:30
6
$\begingroup$

Here's my approach to notebooks:

Reverse @ Sort @ Counts @ Flatten @ Map[
  Cases[
     NotebookImport[#, "Input" -> "HeldExpression"],
     s_Symbol /; StringStartsQ[SymbolName@Unevaluated@s, _?UpperCaseQ],
     \[Infinity],
     Heads -> True
  ] &
] @ nbsPathsList

and for packages:

SetAttributes[customHold, HoldAllComplete]; 
   (*to not interfere with Hold's counts*)

data = Cases[
  Catenate[ReadList[#, customHold[Expression]] & /@ files],
  s_ /; StringStartsQ[SymbolName@Unevaluated@s, _?UpperCaseQ] :> s,
  \[Infinity],
  Heads -> True
];

data // Counts // Sort // #[[-35 ;; -2]] & // 
 BarChart[Values[#], ChartLabels -> Normal[#], BarOrigin -> Left, 
   ImageSize -> 1400, BaseStyle -> 12, BarSpacing -> .5] &

doesn't look impressive but I don't have all repositories pulled locally at home :P

enter image description here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.