Frequency of punctuation characters used in Mathematica languages

Question

I would like to determine the computer language character frequency in Mathematica code. I want to remap my keyboard to the most used character such as [], {}, -, +.

There are the frequency of punctuation characters reported for many progamming languages: C, C++, Java...

Could we report our Mathematica code to dertermine the frequency? So what is the most frequent character in Mathematica?

ALL LANGUAGES

Source: http://xahlee.info/comp/computer_language_char_distribution.html

Answer 1

Like belisarius said in a comment, it's going to be a lot more accurate if you check the frequency on your own codebase. This is how to check the frequency of special characters in the 100 most popular answers on this site.

Needs["jSoupLink`"] (* Package: http://mathematica.stackexchange.com/questions/71914/extract-information-from-html-using-css-selectors *)
url = "https://api.stackexchange.com/2.2/answers?pagesize=100&order=desc&sort=votes&site=mathematica&filter=!1zSk*x-JSjj*nMuua98Vm";
data = Import[url, "JSON"];
data = "body" /. ("items" /. data);
answers = ParseHTMLFragment[#, "code", {"text"}] & /@ data;
counts = Tally@Characters[StringJoin @@ Flatten@answers];
counts = Reverse@SortBy[counts, Last];
    notSpecial = Join[
   CharacterRange["A", "Z"],
   CharacterRange["a", "z"],
   CharacterRange["0", "9"],
   {" "}
   ];
DeleteCases[counts, {Alternatives @@ notSpecial, _}] // TableForm

If any user here wants to check their own character frequency, you can generate a URL that collects your answers by providing your user ID like this:

url[id_] := "https://api.stackexchange.com/2.2/users/" <> ToString@id <> "/answers?pagesize=100&order=desc&sort=activity&site=mathematica&filter=!4)7xaKyZCTSLfUVAL";
data = Import[url[yourUserId], "JSON"];

Answer 2

(* select your codebase files *)
SetDirectory[$InstallationDirectory];
fns = FileNames["*.nb", {"*"}, Infinity][[1 ;; 10]]
(* Read files and tally the chars *)
pp = SortBy[Tally@Flatten@(Characters[Import[#, "Text"]] & /@ fns), -#[[2]] &]
(* discard the letters & digits *)
pp1 = Select[pp, ! StringMatchQ[#[[1]], WordCharacter] &]
(* Plot *)
BarChart[pp1[[All, 2]], ChartLabels -> pp1[[All, 1]]]

Note: the first two are "space" and "newline"

Answer 3

Applying belisarius's code to look at all included package files with the modification of:

fns = FileNames["*.m", {"*"}, Infinity];

I get:

Not much love for Slot-based functions it seems.

Answer 4

@belisarius is right to point out that the frequency strongly depends on the programming style. The answer reported above are mostly applied for .m file (package) file, where there are many comments (* *), many strings "", such as in this code.

(* Mathematica Package *)
(* Created by the Wolfram Workbench 11-Jan-2010 *)    
(*
  TODO
    Way to pass down options with GenerateCode
    Support for unknown expressions passed into CForm
*)    
BeginPackage["SymbolicC`"]    
ToCCodeString::usage = "ToCCodeString[ symbolicC] generates a string of C code from a symbolic C expression."

That's why, the string " and * is really frequent in packages. But I want to check the character frequency in normal Mathematica example, not in packages, such as demonstrations, or fast-code example. This is the results from many featured demonstrations in Wolfram Demonstration project.

ALL LANGUAGES

Source: http://xahlee.info/comp/computer_language_char_distribution.html

MATHEMATICA

Discussions:

The , and [, ] is the most frequent character. Due to the function, argument style [argu1, argu2, argu3]

0, 1, 2 is mainly used in Manipulate[...,{x,0,n}]. In application demonstrations, number is very frequent.

{} is really important for list, and in Manipulate[expr,{u,umin,umax}]

The quote " are mainly used for interface such as Row, Style, Text, Labels, Options, Appearances ...

= is important (Set)

/ and . are used in ReplaceAll /.

; is less frequent

> is more frequent than <, used in :>, ->

_ and : are mainly used for function declaration but less frequent. In daily Mathematica use, no need to delaire new function :)

Surprisingly, @, &, # are really important in functional programming, but less frequent in demonstrations.

Remap keyboard Finally, I will remap my keyboard for the [],{} using with my thumb, index finger in Mathematica because these are the most used characters. For numbers, I will use the numeric keyboard because pressing 0 with right pinky finger is painful.

Answer 5

Here's a way to extend belisarius's method to compare two code bases, by looking at the percent-wise distribution of punctuation characters:

codeBaseCounts[files_] :=

 KeySelect[Not@*StringMatchQ[WhitespaceCharacter | WordCharacter]]@
    AssociationThread[
     Map[FromCharacterCode]@Keys[#],
     Values[#]/Total[Values[#]]
     ] &@SortBy[
   Counts@Flatten@(ToCharacterCode@Import[#, "Text"] & /@ files), -# &]

codeBaseCountsPlot[assoc_, ops___?OptionQ] :=

 BarChart[Values@assoc, ops, ChartLabels -> Keys@assoc]

codingStyleCompare[files1_List, files2_List] :=

 With[{base1 = codeBaseCounts[files1], base2 = codeBaseCounts[files2]},
  Grid[{
    {
     codeBaseCountsPlot[
      Association@
       KeyValueMap[
        # -> #2 - base2[#] &,
        base1
        ],
      ImageSize -> 500
      ],
     SpanFromLeft
     },
    {
     codeBaseCountsPlot[base1, ImageSize -> 250],
     codeBaseCountsPlot[base2, ImageSize -> 250]
     }
    },
   ItemSize -> Full
   ]
  ]

This lets up do things like this:

codingStyleCompare[
 FileNames["*.m", $UserBasePacletsDirectory, Infinity],
 FileNames["*.m", $InstallationDirectory, Infinity]
 ]

which shows, I think, that the $UserBasePacletsDirectory contains more code, as the number of square-brackets increases as a percentage, representing more calls. It also contains many more string characters, though, which is interesting. Probably from the PacletInfo.m files.