13
$\begingroup$

I would like to determine the computer language character frequency in Mathematica code. I want to remap my keyboard to the most used character such as [], {}, -, +.

There are the frequency of punctuation characters reported for many progamming languages: C, C++, Java...

Could we report our Mathematica code to dertermine the frequency? So what is the most frequent character in Mathematica?

ALL LANGUAGES enter image description here

Source: http://xahlee.info/comp/computer_language_char_distribution.html

$\endgroup$
  • 4
    $\begingroup$ I guess that strongly depends on your programming style (i.e. f[x] vs. f@x vs x // f ,or the improbable x~f~Sequence[ ], {a, b} vs. List[a, b]etc). So you first need your codebase $\endgroup$ – Dr. belisarius May 29 '15 at 15:31
  • 2
    $\begingroup$ Also, numbers aren't usually considered "punctuation" or "special" characters! $\endgroup$ – Dr. belisarius May 29 '15 at 15:41
  • $\begingroup$ The documentation center might prove a useful mine for WL code. $\endgroup$ – chuy May 29 '15 at 16:20
  • $\begingroup$ @belisarius I think I recall that in certain golf-oriented languages numbers are used as operators. $\endgroup$ – Mr.Wizard May 29 '15 at 16:35
  • 3
    $\begingroup$ I think this question verges on "primarily opinion-based Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise." Unless someone can come up with an authoritative reference I will probably convert this to a community wiki as a compromise. $\endgroup$ – Mr.Wizard May 29 '15 at 16:38
14
$\begingroup$

Like belisarius said in a comment, it's going to be a lot more accurate if you check the frequency on your own codebase. This is how to check the frequency of special characters in the 100 most popular answers on this site.

Needs["jSoupLink`"] (* Package: http://mathematica.stackexchange.com/questions/71914/extract-information-from-html-using-css-selectors *)
url = "https://api.stackexchange.com/2.2/answers?pagesize=100&order=desc&sort=votes&site=mathematica&filter=!1zSk*x-JSjj*nMuua98Vm";
data = Import[url, "JSON"];
data = "body" /. ("items" /. data);
answers = ParseHTMLFragment[#, "code", {"text"}] & /@ data;
counts = Tally@Characters[StringJoin @@ Flatten@answers];
counts = Reverse@SortBy[counts, Last];
    notSpecial = Join[
   CharacterRange["A", "Z"],
   CharacterRange["a", "z"],
   CharacterRange["0", "9"],
   {" "}
   ];
DeleteCases[counts, {Alternatives @@ notSpecial, _}] // TableForm

Mathematica graphics

If any user here wants to check their own character frequency, you can generate a URL that collects your answers by providing your user ID like this:

url[id_] := "https://api.stackexchange.com/2.2/users/" <> ToString@id <> "/answers?pagesize=100&order=desc&sort=activity&site=mathematica&filter=!4)7xaKyZCTSLfUVAL";
data = Import[url[yourUserId], "JSON"];
$\endgroup$
  • 4
    $\begingroup$ Cool, the top-100 posts don't have unmatched brackets! However, the number of double quotes is odd, which is odd (but not impossible). $\endgroup$ – Sjoerd C. de Vries May 29 '15 at 16:13
  • $\begingroup$ @SjoerdC.deVries The bracket matching fails in my codebase :) $\endgroup$ – Dr. belisarius May 29 '15 at 16:22
  • 3
    $\begingroup$ @SjoerdC.deVries Escaped quote perhaps?: "\"" $\endgroup$ – Mr.Wizard May 29 '15 at 16:34
  • 1
    $\begingroup$ A select few wizards, you mean? $\endgroup$ – Sjoerd C. de Vries May 29 '15 at 18:41
  • 1
    $\begingroup$ @SjoerdC.deVries I checked the top 100 answers for each of the top 5 users and in those answers they used tilde 52 (Mr. W.), 0, 4, 3 and 28 (L.S.) times respectively. :) $\endgroup$ – C. E. May 29 '15 at 21:59
5
$\begingroup$
(* select your codebase files *)
SetDirectory[$InstallationDirectory];
fns = FileNames["*.nb", {"*"}, Infinity][[1 ;; 10]]
(* Read files and tally the chars *)
pp = SortBy[Tally@Flatten@(Characters[Import[#, "Text"]] & /@ fns), -#[[2]] &]
(* discard the letters & digits *)
pp1 = Select[pp, ! StringMatchQ[#[[1]], WordCharacter] &]
(* Plot *)
BarChart[pp1[[All, 2]], ChartLabels -> pp1[[All, 1]]]

Mathematica graphics

Note: the first two are "space" and "newline"

$\endgroup$
  • $\begingroup$ is that representative code? Odd for & , # to be low and ? so high.. $\endgroup$ – george2079 May 29 '15 at 16:24
  • $\begingroup$ @george2079 added the directory name :) $\endgroup$ – Dr. belisarius May 29 '15 at 16:25
  • 1
    $\begingroup$ I see, this is importing the entire low level notebook. Properly you need to just include the text of input cells.. $\endgroup$ – george2079 May 29 '15 at 16:42
  • $\begingroup$ @george Good point! belisarius, why not look at .m files instead? $\endgroup$ – Mr.Wizard May 29 '15 at 16:47
  • $\begingroup$ Ashamed and pondering about to do :) $\endgroup$ – Dr. belisarius May 29 '15 at 16:50
3
$\begingroup$

Applying belisarius's code to look at all included package files with the modification of:

fns = FileNames["*.m", {"*"}, Infinity];

I get:

enter image description here

Not much love for Slot-based functions it seems.

$\endgroup$
  • $\begingroup$ If I had to guess (just spitballing off of your note about Slot), it's that these files are generally source files like [documentedContexts.m[(mathematica.stackexchange.com/a/151967/38205) which are lists of properties, file paths, etc. Plus some framework stuff, which is chaining small functions. Trying $UserBasePacletsDirectory gives very different, probably more representative, results. $\endgroup$ – b3m2a1 Aug 4 '17 at 22:26
2
$\begingroup$

@belisarius is right to point out that the frequency strongly depends on the programming style. The answer reported above are mostly applied for .m file (package) file, where there are many comments (* *), many strings "", such as in this code.

(* Mathematica Package *)
(* Created by the Wolfram Workbench 11-Jan-2010 *)    
(*
  TODO
    Way to pass down options with GenerateCode
    Support for unknown expressions passed into CForm
*)    
BeginPackage["SymbolicC`"]    
ToCCodeString::usage = "ToCCodeString[ symbolicC] generates a string of C code from a symbolic C expression."

That's why, the string " and * is really frequent in packages. But I want to check the character frequency in normal Mathematica example, not in packages, such as demonstrations, or fast-code example. This is the results from many featured demonstrations in Wolfram Demonstration project.

ALL LANGUAGES enter image description here

Source: http://xahlee.info/comp/computer_language_char_distribution.html

MATHEMATICA enter image description here

Discussions:

The , and [, ] is the most frequent character. Due to the function, argument style [argu1, argu2, argu3]

0, 1, 2 is mainly used in Manipulate[...,{x,0,n}]. In application demonstrations, number is very frequent.

{} is really important for list, and in Manipulate[expr,{u,umin,umax}]

The quote " are mainly used for interface such as Row, Style, Text, Labels, Options, Appearances ...

= is important (Set)

/ and . are used in ReplaceAll /.

; is less frequent

> is more frequent than <, used in :>, ->

_ and : are mainly used for function declaration but less frequent. In daily Mathematica use, no need to delaire new function :)

Surprisingly, @, &, # are really important in functional programming, but less frequent in demonstrations.

Remap keyboard Finally, I will remap my keyboard for the [],{} using with my thumb, index finger in Mathematica because these are the most used characters. For numbers, I will use the numeric keyboard because pressing 0 with right pinky finger is painful.

$\endgroup$
0
$\begingroup$

Here's a way to extend belisarius's method to compare two code bases, by looking at the percent-wise distribution of punctuation characters:

codeBaseCounts[files_] :=

 KeySelect[Not@*StringMatchQ[WhitespaceCharacter | WordCharacter]]@
    AssociationThread[
     Map[FromCharacterCode]@Keys[#],
     Values[#]/Total[Values[#]]
     ] &@SortBy[
   Counts@Flatten@(ToCharacterCode@Import[#, "Text"] & /@ files), -# &]

codeBaseCountsPlot[assoc_, ops___?OptionQ] :=

 BarChart[Values@assoc, ops, ChartLabels -> Keys@assoc]

codingStyleCompare[files1_List, files2_List] :=

 With[{base1 = codeBaseCounts[files1], base2 = codeBaseCounts[files2]},
  Grid[{
    {
     codeBaseCountsPlot[
      Association@
       KeyValueMap[
        # -> #2 - base2[#] &,
        base1
        ],
      ImageSize -> 500
      ],
     SpanFromLeft
     },
    {
     codeBaseCountsPlot[base1, ImageSize -> 250],
     codeBaseCountsPlot[base2, ImageSize -> 250]
     }
    },
   ItemSize -> Full
   ]
  ]

This lets up do things like this:

codingStyleCompare[
 FileNames["*.m", $UserBasePacletsDirectory, Infinity],
 FileNames["*.m", $InstallationDirectory, Infinity]
 ]

compare

which shows, I think, that the $UserBasePacletsDirectory contains more code, as the number of square-brackets increases as a percentage, representing more calls. It also contains many more string characters, though, which is interesting. Probably from the PacletInfo.m files.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.