# Haiku Extractor

Inspired by the cool Tumblr Haiku Times that searches for accidental haiku in New York Times articles, I tried my hand at implementing such a search in Mathematica for my own text samples.

I'm still not a one-liner writer, so I'd welcome any input on more efficient approaches than mine below (especially the Table in Step 4!), but to follow the rules my question is how to implement my Step 6 automatically as I'm doing it manually now:

sampleText =  "Haikus are easy but sometimes they don't make sense refrigerator. Make a man a king today and tomorrow he will be a brigand. Dragonflies, by contrast, look dainty, glittery and fun, like a bubble bath or costume jewelry, and they're often grouped with butterflies and ladybugs on the very short list of Insects People Like.";


Step 2. Break up paragraphs into sentences at periods

sampleSentences = StringSplit[sampleText,"."]


Step 3. Break up sentences into words

sampleWords = StringSplit[sampleSentences]


Step 4. Count each sentence's syllables

syllableCount[x_] := Length[WordData[x, "Hyphenation"]]
sampleSyllables =
Table[Apply[Plus,
Map[syllableCount, ToLowerCase[sampleWords[[i]]]]], {i, 1,
Length[sampleWords]}]

{16, 17, 41}


Step 5. Select the sentences with 17 syllables

sampleHaiku =
sampleSentences[[Flatten[Position[sampleSyllables, 17]]]]

{" Make a man a king today and tomorrow he will be a brigand"}


Ironically:

syllableCount["haikus"]

1


Step 6. Break the 17-syllabic ones up at 5, 7, 5 and display the result
(I ran out of ideas here...)

Thanks in advance and apologies for protocol/formatting gaffes on my first post!

-
It's a fine question and well posted in my opinion. –  BoLe Apr 3 '13 at 11:20
WordData words listing isn't very comprehensive. Try WordData["haikus", "Hyphenation"]. But haiku works, though –  belisarius Apr 3 '13 at 11:30
English "written" syllables therefore do not correspond to the actually spoken syllables of the living language (Wikipedia). Might that imply that hyphenation data is not the correct method to count syllables in a haiku ( which is intended to be read aloud)? –  Sjoerd C. de Vries Apr 3 '13 at 14:20

As belisarius noted in a comment above, WordData can sometimes be inadequate. It might be more robust (but slower) if we use Wolfram | Alpha from inside Mathematica to retrive the syllable count with the following function:

ClearAll@syllables
SetAttributes[syllables, Listable]
syllables[word_String] := Length@WolframAlpha[
"syllables " <> word, {{"Hyphenation:WordData", 1}, "ComputableData"}]


Here's a haiku splitter based on the above function. If a haiku is not possible, it returns $Failed: haikuSplit[str_String] := With[{s = Accumulate@syllables@#}, With[{l = s /. {a___, 5, b___, 12, c___, 17} :> Length /@ {{a}, {b}, {c}} + 1}, If[l === s, Print["No haiku possible!"];$Failed,
Composition[StringJoin, Riffle[#, " "] &] /@
InternalPartitionRagged[#, l] // TableForm
]
]
] &@StringSplit[str, Whitespace]


Now let's test this:

haikuSplit["Make a man a king today and tomorrow he will be a brigand"]
(*  Make a man a king
today and tomorrow he
will be a brigand *)

haikuSplit["Make a man a king today and tomorrow he will be a haiku expert"]
(*  No haiku possible!
\$Failed *)

-
WolframAlpha["syllables dragonflies",{{"Hyphenation:WordData", 1}, "ComputableData"}] misses data, yet W|A does recompute with "dragonfly" and returns its hyphenation. WolframAlpha["hyphenate " <> "dragonflies", "PodCells"]. It's the same number of syllables. –  BoLe Apr 3 '13 at 17:18
Hmm... it's probably because W|A interprets the plural form of the animal/plant name as referring to the class of animals instead of just the word. It also fails with lions, tigers, cats, dogs, elephants,... I don't use W|A all that much, so I am not sure if it is possible to force it to interpret it as a word instead. –  rm -rf Apr 3 '13 at 17:58
It fails here and there, yes. How complex would you say the engine behind the mentioned Times Haiku is? Such a tiny, yet fascinating blog. –  BoLe Apr 3 '13 at 18:11
I'm sure the NYT engine is written in something like perl. There are several advanced packages available for NLP in perl (e.g. see this) and it probably is no more than half a page or 2 at most of code. Even those might not be 100% accurate (although the situations in which they fail are different). Syllable counting is not easy and English, being a mongrel language, makes things much harder than say, for German. Of course, the problem with German is that you hit 17 syllables before the second word, thus making haikus impossible ;) –  rm -rf Apr 3 '13 at 18:17
Could one utilize LaTeX hyphenating in Mathematica instead of WordData or W|A (which share database I guess), would that be very hard to achieve? Since it seems very powerful to me. (And a package for my mother tongue, Slovene exists for bonus.) –  BoLe Apr 3 '13 at 18:35

Step 2. Append an empty space to dot.

sampleSentences = StringSplit[sampleText, ". "]


Step 4. Function syllableCount defined by you.

Total /@ Map[syllableCount, StringSplit[sampleSentences], {2}]


{16, 17, 40}

Step 5. The 17-syllable sentences.

h = Select[sampleSentences,
Total[syllableCount /@ StringSplit[#]] == 17 &]


{"Make a man a king today and tomorrow he will be a brigand"}

Step 6. Breaking of a string in three 5-, 7- and 5-syllable strings.

cut[haiku_] :=
Module[{words = StringSplit[haiku, " "], h2},
h2 = Accumulate[syllableCount /@ words];
Apply[StringJoin, Riffle[Take[words, #], " "]] & /@
(Position[h2, 5 | 12] /. {{i_}, {j_}} :>
{{1, i}, {i + 1, j}, {j + 1, -1}})]


Breaking selected sentences.

cut /@ h


{{"Make a man a king", "today and tomorrow he", "will be a brigand"}}

StringJoin[Riffle[cut@#, "\n"] & /@ h]


Make a man a king
today and tomorrow he
will be a brigand

-
Please try WordData[#, "Hyphenation"] & /@ ToLowerCase[sampleWords[[2]]] . Your result is right, but only by chance –  belisarius Apr 3 '13 at 12:06
This still returns some Missing["NotAvailable"] elements albeit fewer. I saw that but I figured only one syllable words are missing hyphenation data (??), and syllableCount measures them correctly as an element list. –  BoLe Apr 3 '13 at 12:18
Not the case. Try WordData["Haikus","Hyphenation"] –  belisarius Apr 3 '13 at 12:25
Oops, I noticed a mistake with arguments for Take in cut. I fixed it with an explicit rule. –  BoLe Apr 3 '13 at 12:35
"Haikus"` is the first word in the first sentence in the question. Sorry. :D –  belisarius Apr 3 '13 at 12:43