# Frequency of punctuation characters used in Mathematica languages

I would like to determine the computer language character frequency in Mathematica code. I want to remap my keyboard to the most used character such as [], {}, -, +.

There are the frequency of punctuation characters reported for many progamming languages: C, C++, Java...

Could we report our Mathematica code to dertermine the frequency? So what is the most frequent character in Mathematica?

ALL LANGUAGES

• I guess that strongly depends on your programming style (i.e. f[x] vs. f@x vs x // f ,or the improbable x~f~Sequence[ ], {a, b} vs. List[a, b]etc). So you first need your codebase May 29, 2015 at 15:31
• Also, numbers aren't usually considered "punctuation" or "special" characters! May 29, 2015 at 15:41
• The documentation center might prove a useful mine for WL code.
– chuy
May 29, 2015 at 16:20
• @belisarius I think I recall that in certain golf-oriented languages numbers are used as operators. May 29, 2015 at 16:35
• I think this question verges on "primarily opinion-based Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise." Unless someone can come up with an authoritative reference I will probably convert this to a community wiki as a compromise. May 29, 2015 at 16:38

Like belisarius said in a comment, it's going to be a lot more accurate if you check the frequency on your own codebase. This is how to check the frequency of special characters in the 100 most popular answers on this site.

Needs["jSoupLink"] (* Package: http://mathematica.stackexchange.com/questions/71914/extract-information-from-html-using-css-selectors *)
data = Import[url, "JSON"];
data = "body" /. ("items" /. data);
answers = ParseHTMLFragment[#, "code", {"text"}] & /@ data;
counts = Tally@Characters[StringJoin @@ Flatten@answers];
counts = Reverse@SortBy[counts, Last];
notSpecial = Join[
CharacterRange["A", "Z"],
CharacterRange["a", "z"],
CharacterRange["0", "9"],
{" "}
];
DeleteCases[counts, {Alternatives @@ notSpecial, _}] // TableForm


If any user here wants to check their own character frequency, you can generate a URL that collects your answers by providing your user ID like this:

url[id_] := "https://api.stackexchange.com/2.2/users/" <> ToString@id <> "/answers?pagesize=100&order=desc&sort=activity&site=mathematica&filter=!4)7xaKyZCTSLfUVAL";
data = Import[url[yourUserId], "JSON"];

• Cool, the top-100 posts don't have unmatched brackets! However, the number of double quotes is odd, which is odd (but not impossible). May 29, 2015 at 16:13
• @SjoerdC.deVries The bracket matching fails in my codebase :) May 29, 2015 at 16:22
• @SjoerdC.deVries Escaped quote perhaps?: "\"" May 29, 2015 at 16:34
• A select few wizards, you mean? May 29, 2015 at 18:41
• @SjoerdC.deVries I checked the top 100 answers for each of the top 5 users and in those answers they used tilde 52 (Mr. W.), 0, 4, 3 and 28 (L.S.) times respectively. :) May 29, 2015 at 21:59
(* select your codebase files *)
SetDirectory[$InstallationDirectory]; fns = FileNames["*.nb", {"*"}, Infinity][[1 ;; 10]] (* Read files and tally the chars *) pp = SortBy[Tally@Flatten@(Characters[Import[#, "Text"]] & /@ fns), -#[[2]] &] (* discard the letters & digits *) pp1 = Select[pp, ! StringMatchQ[#[[1]], WordCharacter] &] (* Plot *) BarChart[pp1[[All, 2]], ChartLabels -> pp1[[All, 1]]]  Note: the first two are "space" and "newline" • is that representative code? Odd for & , # to be low and ? so high.. May 29, 2015 at 16:24 • @george2079 added the directory name :) May 29, 2015 at 16:25 • I see, this is importing the entire low level notebook. Properly you need to just include the text of input cells.. May 29, 2015 at 16:42 • @george Good point! belisarius, why not look at .m files instead? May 29, 2015 at 16:47 • Ashamed and pondering about to do :) May 29, 2015 at 16:50 Applying belisarius's code to look at all included package files with the modification of: fns = FileNames["*.m", {"*"}, Infinity];  I get: Not much love for Slot-based functions it seems. • If I had to guess (just spitballing off of your note about Slot), it's that these files are generally source files like [documentedContexts.m[(mathematica.stackexchange.com/a/151967/38205) which are lists of properties, file paths, etc. Plus some framework stuff, which is chaining small functions. Trying $UserBasePacletsDirectory gives very different, probably more representative, results. Aug 4, 2017 at 22:26

@belisarius is right to point out that the frequency strongly depends on the programming style. The answer reported above are mostly applied for .m file (package) file, where there are many comments (* *), many strings "", such as in this code.

(* Mathematica Package *)
(* Created by the Wolfram Workbench 11-Jan-2010 *)
(*
TODO
Way to pass down options with GenerateCode
Support for unknown expressions passed into CForm
*)
BeginPackage["SymbolicC"]
ToCCodeString::usage = "ToCCodeString[ symbolicC] generates a string of C code from a symbolic C expression."


That's why, the string " and * is really frequent in packages. But I want to check the character frequency in normal Mathematica example, not in packages, such as demonstrations, or fast-code example. This is the results from many featured demonstrations in Wolfram Demonstration project.

ALL LANGUAGES

MATHEMATICA

Discussions:

The , and [, ] is the most frequent character. Due to the function, argument style [argu1, argu2, argu3]

0, 1, 2 is mainly used in Manipulate[...,{x,0,n}]. In application demonstrations, number is very frequent.

{} is really important for list, and in Manipulate[expr,{u,umin,umax}]

The quote " are mainly used for interface such as Row, Style, Text, Labels, Options, Appearances ...

= is important (Set)

/ and . are used in ReplaceAll /.

; is less frequent

> is more frequent than <, used in :>, ->

_ and : are mainly used for function declaration but less frequent. In daily Mathematica use, no need to delaire new function :)

Surprisingly, @, &, # are really important in functional programming, but less frequent in demonstrations.

Remap keyboard Finally, I will remap my keyboard for the [],{} using with my thumb, index finger in Mathematica because these are the most used characters. For numbers, I will use the numeric keyboard because pressing 0 with right pinky finger is painful.

Here's a way to extend belisarius's method to compare two code bases, by looking at the percent-wise distribution of punctuation characters:

codeBaseCounts[files_] :=

KeySelect[Not@*StringMatchQ[WhitespaceCharacter | WordCharacter]]@
Map[FromCharacterCode]@Keys[#],
Values[#]/Total[Values[#]]
] &@SortBy[
Counts@Flatten@(ToCharacterCode@Import[#, "Text"] & /@ files), -# &]

codeBaseCountsPlot[assoc_, ops___?OptionQ] :=

BarChart[Values@assoc, ops, ChartLabels -> Keys@assoc]

codingStyleCompare[files1_List, files2_List] :=

With[{base1 = codeBaseCounts[files1], base2 = codeBaseCounts[files2]},
Grid[{
{
codeBaseCountsPlot[
Association@
KeyValueMap[
# -> #2 - base2[#] &,
base1
],
ImageSize -> 500
],
SpanFromLeft
},
{
codeBaseCountsPlot[base1, ImageSize -> 250],
codeBaseCountsPlot[base2, ImageSize -> 250]
}
},
ItemSize -> Full
]
]


This lets up do things like this:

codingStyleCompare[
FileNames["*.m", $UserBasePacletsDirectory, Infinity], FileNames["*.m",$InstallationDirectory, Infinity]
]


which shows, I think, that the \$UserBasePacletsDirectory contains more code, as the number of square-brackets increases as a percentage, representing more calls. It also contains many more string characters, though, which is interesting. Probably from the PacletInfo.m files.