10
$\begingroup$

Consider the following:

StringData1={"Dog","Dog","Dog","Dog","Duck","Duck","Duck","Rabbit","House"};
StringData2={"A dog is in the house.","A dog is outside.",
    "Why is everybody talking about a dog?", "Dog", "The duck died in the house.", 
    "The duck is actually a rabbit.", "Duck", "And the rabbit in fact is a dog", 
    "I have a dog in my house."};

Using the following function, it is quite easy to plot a histogram for StringData1. However the options remain fixed.

StringHistogram[list_] :=
 Module[{counter, strings, numbers},
  counter = {First@#, Length@#} & /@ (GatherBy@list);
  strings = First /@ counter;
  numbers = Last /@ counter;

  BarChart[numbers, ChartLabels -> strings]
  ]

In case of StringData2 I was wondering whether it would be possible to chop up each sentence into single words (e.g.{Sentence1,...} -> {{Word1, Word2,...},...}) and then to run StringHistogram on Flatten@StringData2New, whereas StringData2New is of the form {{Word1, Word2,...},...}.

$\endgroup$

4 Answers 4

13
$\begingroup$

I have a slightly different strategy for splittling the string than rcollyer, because you can stick the pattern into StringSplit directly instead of needing to do a StringReplace first to prepare the string for splitting. For instance:

In[141]:= StringSplit["A dog is in the house.", Except[WordCharacter]]
Out[141]= {"A", "dog", "is", "in", "the", "house"}

Also, you can make StringHistogram simpler by using the built-in Tally function, which you've reimplemented (pretty darn well, I must say):

StringHistogram[list_, opts : OptionsPattern[]] :=
 With[{tally = Tally@list},
  BarChart[tally[[All, 2]],
   BarOrigin -> Left,
   ChartLabels -> tally[[All, 1]],
   FilterRules[{opts}, Options[BarChart]]
   ]]

I use the BarOrigin option to make the bars come from the side, which makes everything much easier to read when you have many strings, and I pass options to BarChart in order to make tweaking things easier:

StringHistogram[
 StringSplit[StringData2, Except[WordCharacter]] // Flatten, 
 ImageSize -> 500, BaseStyle -> "Label"]

big histogram

$\endgroup$
1
  • 1
    $\begingroup$ Except[WordCharacter] is a very good alternative, and I'm ashamed to admit I never think of Except, +1. $\endgroup$
    – rcollyer
    Apr 27, 2012 at 21:51
9
$\begingroup$

You're looking for StringSplit. In your case, I would do this

StringSplit[{"A dog is in the house.","A dog is outside."}]

which returns

(* {{"A", "dog", "is", "in", "the", "house."}, 
    {"A", "dog", "is", "outside."}} *)

Unfortunately, that leaves in the punctuation. To rid yourself of the punctuation, I would use StringReplace first, as follows

StringSplit @ StringReplace[{"A dog is in the house.", "A dog is outside."},
  RegularExpression["[[:punct:]]"]

which gives

(* {{"A", "dog", "is", "in", "the", "house"}, 
    {"A", "dog", "is", "outside"}} *)

Note the use of the character class in the RegularExpression; it makes specifying all punctuation easier.

$\endgroup$
6
$\begingroup$

Regarding the first part of your question, I would rewrite StringHistogram like this:

StringHistogram[list_] := BarChart[#2, ChartLabels -> #1] & @@ Transpose[Tally@list]

If you want to supply additional options, you can extend this as:

StringHistogram[list_, opts : OptionsPattern[]] := 
    BarChart[#2, ChartLabels -> #1, opts] & @@ Transpose[Tally@list]

I think Pillsy's suggestion to split the strings is very clean. Here's another equally clean approach using StringCases:

StringCases[StringData2, x : WordCharacter .. :> x]
$\endgroup$
2
$\begingroup$

The TextWords feature was introduced in 2015. TextWords connects to the internet so that is a possible limitation.

enter image description here


Clear["Global`*"];
StringData1 = {"Dog", "Dog", "Dog", "Dog", "Duck", "Duck", "Duck", 
   "Rabbit", "House"};
StringData2 = {"A dog is in the house.", "A dog is outside.", 
   "Why is everybody talking about a dog?", "Dog", 
   "The duck died in the house.", "The duck is actually a rabbit.", 
   "Duck", "And the rabbit in fact is a dog", 
   "I have a dog in my house."};
tc1 = TextWords[StringData2] // Flatten // DeleteStopwords // 
    ToLowerCase // Tally // ReverseSortBy[Last][#] &
BarChart[tc1[[All, 2]], BarOrigin -> Left, 
 ChartLabels -> tc1[[All, 1]]]

Users can compare results by excluding DeleteStopWords from the above.

enter image description here


StringSplit was updated in 2020. Can't say exactly what was updated but another syntax can be used for such cases that was most likely not present in 2012.

Flatten@StringSplit[
  StringData2, {PunctuationCharacter, WhitespaceCharacter}]

{"A", "dog", "is", "in", "the", "house", "A", "dog", "is", "outside", \ "Why", "is", "everybody", "talking", "about", "a", "dog", "Dog",
"The", "duck", "died", "in", "the", "house", "The", "duck", "is",
"actually", "a", "rabbit", "Duck", "And", "the", "rabbit", "in",
"fact", "is", "a", "dog", "I", "have", "a", "dog", "in", "my",
"house"}

and

Flatten@TextWords[StringData2] == 
 Flatten@StringSplit[
   StringData2, {PunctuationCharacter, WhitespaceCharacter}]

True

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.