1
$\begingroup$

I generated word clouds from a text file in Mathematica. I was able to remove words with 5 or less characters from the cloud.

 fdp2 = StringReplace[fdp, 
 WordBoundary ~~ Repeated[WordCharacter, 5] ~~ WordBoundary -> ""]
 WordCloud[fdp2]

But there are still useless words in the cloud. How can I remove a specific word like "become" from a word cloud?

$\endgroup$
2
  • $\begingroup$ could you please provide code for fdp? $\endgroup$ Nov 30, 2015 at 9:55
  • 2
    $\begingroup$ From a word cloud or from the string source? There is StringDelete now but you can proceed with your current method too, what exactly is the problem? $\endgroup$
    – Kuba
    Nov 30, 2015 at 10:24

1 Answer 1

6
$\begingroup$

If you building a word cloud there are a few tricks. First of all use DeleteStopwords. Compare:

cat = Import["https://en.wikipedia.org/wiki/Cat"];

{WordCloud[cat], WordCloud[DeleteStopwords[cat]]}

enter image description here

Yes, the right one much better, but now we see the next problem "cat" vs "cats". I usually use reduce to baseform:

Clear@base;
base[w_] := With[
   {tmp = WordData[w, "BaseForm", "List"]},
   If[(Head[tmp] === Missing) || tmp === {}, w, tmp[[1]]]];
SetAttributes[base, Listable];

I would also consider removing numbers (not a must though) and black-listing. For example Wikipedia pages often noisy with:

blackLIST = {"doi", "ed", "isbn", "pmid"};

I would also use ScalingFunctions -> (#^s &) where $0<s<5$ is a correction that emphasizes or deemphasizes word frequency-size visual (5 is a good number, could be more though). Here we go:

WordCloud[
 DeleteCases[
  base[TextWords[StringDelete[DeleteStopwords[ToLowerCase[cat]], 
  DigitCharacter ..]]],
  Alternatives @@ blackLIST],
 ScalingFunctions -> (#^.3 &)]

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.