4
$\begingroup$

There are interesting histograms about the English language:

BarChart[Counts[StringTake[WordList[], 1]], ChartLabels -> Automatic]

enter image description here

Histogram[StringLength[WordList[]]]

enter image description here

I want to do something like this with the Holy Bible.

I tried:

numberofwords = 
Take[WolframAlpha[
"Bible", {{"Word:FamousTextData", 1}, "ComputableData"}], 1]

enter image description here

There are lists of the most frequent words:

WolframAlpha["Bible", IncludePods -> "MostCommonWords:FamousTextData",
AppearanceElements -> {"Pods"}, 
TimeConstraint -> {30, Automatic, Automatic, Automatic}]

enter image description here

But I don't know how to determine the most frequent letters (alphabet).

How could I do this by using Mathematica?

Thank you in advance to any one who may be able to give me some ideas

$\endgroup$
5
$\begingroup$

Something like this perhaps?

bibOriginal = 
  Import["http://www.gutenberg.org/cache/epub/10/pg10.txt"]; (* load the bible *)
bib = StringDrop[bibOriginal, 658]; (* drop the introductory text *)
bib = StringDrop[bib, -18733];(* drop the legalistic closeout at the end*)
bibC = Map[{#[[1]], Length[#]} &, Gather[Sort[ToLowerCase[Characters[bib]]]]];
bibC = Select[bibC, MemberQ[CharacterRange["a", "z"], #[[1]]] &]
  (*ditch punctuation and numbers*);
BarChart[bibC[[All, 2]], ChartLabels -> bibC[[All, 1]]]

bible character distribution

TableForm[Sort[bibC, #1[[2]] > #2[[2]] &]]

enter image description here

The same basic approach would probably work with the original text, so long as you were sensitive to the shift from Hebrew to Aramaic where appropriate.

$\endgroup$
  • $\begingroup$ Nice. I knew I was taking too long to work out the answer :-) $\endgroup$ – Jason B. Mar 21 '16 at 15:38
4
$\begingroup$

We can import the bible directly from the web and convert it to a list of words fairly easily. Here I'm going to remove the chapter/verse headings as well as extraneous punctuation marks.

biblewords = 
  Import["http://www.gutenberg.org/cache/epub/10/pg10.txt"] // 
          StringSplit[#, {"*** START OF THIS PROJECT GUTENBERG EBOOK THE KING JAMES BIBLE ***", 
             "*** END OF THIS PROJECT GUTENBERG EBOOK THE KING JAMES BIBLE ***"}] & //
         StringTrim //
        StringSplit[#, {" ", ".", "\n", ","}][[2]] & //
        Flatten //
        DeleteCases[#, ""] & //
        Select[#, (Not@
          StringMatchQ[#, 
           DigitCharacter .. ~~ ":" ~~ DigitCharacter ..] &)] & //
        StringReplace[#, (Alternatives @@ Characters[",.?:;'\"!-()*"]) -> 
          ""] & // ToLowerCase;

We can ask how many words and how many unique words are in the bible,

Length@biblewords
Length@DeleteDuplicates@biblewords
(* 790146 *)
(* 12703 *)

This page lists 783,137 total words and 12,143 unique words, so the above is pretty close. Any refinements on the above importing scheme are most appreciated. We can use Tally to get the 30 most common words and give their prevalence:

TableForm[
 {#1, #2/Length[biblewords] 100.0} & @@@ (Sort[
     Tally@biblewords, #1[[2]] > #2[[2]] &][[;; 30]]), 
 TableHeadings -> {None, {"Word", "Frequency"}}]

enter image description here

And, finally, the character count,

TableForm[Tally[Characters@StringJoin@biblewords], 
 TableHeadings -> {None, {"Character", "Frequency"}}]

enter image description here

$\endgroup$
  • 2
    $\begingroup$ Try using TextWords to get words and then using DeleteStopwords to remove common words that aren't generally interesting. $\endgroup$ – Searke Mar 21 '16 at 18:32
4
$\begingroup$

I will start by just copying how Jason B imported the bible:

bible = Import["http://www.gutenberg.org/cache/epub/10/pg10.txt"];
bible = StringSplit[bible, {
     "*** START OF THIS PROJECT GUTENBERG EBOOK THE KING JAMES BIBLE \
***", "*** END OF THIS PROJECT GUTENBERG EBOOK THE KING JAMES BIBLE ***"
     }][[2]];

After that I will use WordCounts and LetterCounts, which are two functions that were introduced in Mathematica 10.1:

wordcounts = WordCounts[bible];
lettercounts = LetterCounts[bible];

Then you can do whatever you want with those counts:

top30 = Take[Transpose[{Keys@wordcounts, Values@wordcounts}], 30];
TableForm[top30, TableHeadings -> {None, {"Words", "Count"}}]

Note that WordCount has the option IgnoreCase. If you add IgnoreCases -> True it will in effect convert all letters into lowercase before counting the words. WordCount can also be used to count n-grams.

$\endgroup$
  • $\begingroup$ +1 but there is a few words, wordcounts // Length = 824 $\endgroup$ – vito Jun 8 '16 at 20:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.