2
$\begingroup$

I would like to ask you how to split the following texts into 4 paragraphs. These are two documents, one starts with "It's official...", and the other one with ",CNN...".

text = {"It's official: U.S. President Barack Obama wants lawmakers to weigh \
    in on whether to use military force in Syria.
     Obama sent a letter to the heads of the House and Senate on Saturday \
    night, hours after announcing that he believes military action \
    against Syrian targets is the right step to take over the alleged use \
    of chemical weapons.
     The proposed legislation from Obama asks Congress to approve the use \
    of military force \"to deter, disrupt, prevent and degrade the \
    potential for future uses of chemical weapons or other weapons of \
    mass destruction.\"
     It's a step that is set to turn an international crisis into a \
    fierce domestic political battle.
     There are key questions looming over the debate: What did U.N. \
    weapons inspectors find in Syria? What happens if Congress votes no? \
    And how will the Syrian government react?
     In a televised address from the White House Rose Garden earlier \
    Saturday, the president said he would take his case to Congress, not \
    because he has to -- but because he wants to.
     \"While I believe I have the authority to carry out this military \
    action without specific congressional authorization, I know that the \
    country will be stronger if we take this course, and our actions will \
    be even more effective,\" he said. \"We should have this debate, \
    because the issues are too big for business as usual.\"
     Obama said top congressional leaders had agreed to schedule a debate \
    when the body returns to Washington on September 9. The Senate \
    Foreign Relations Committee will hold a hearing over the matter on \
    Tuesday, Sen. Robert Menendez said.
     Transcript: Read Obama's full remarks
     Syrian crisis: Latest developments
     U.N. inspectors leave Syria
     Obama's remarks came shortly after U.N. inspectors left Syria, \
    carrying evidence that will determine whether chemical weapons were \
    used in an attack early last week in a Damascus suburb.




     ", "(CNN) -- Usain Bolt rounded off the world championships Sunday \
    by claiming his third gold in Moscow as he anchored Jamaica to \
    victory in the men's 4x100m relay.
     The fastest man in the world charged clear of United States rival \
    Justin Gatlin as the Jamaican quartet of Nesta Carter, Kemar \
    Bailey-Cole, Nickel Ashmeade and Bolt won in 37.36 seconds.
     The U.S finished second in 37.56 seconds with Canada taking the \
    bronze after Britain were disqualified for a faulty handover.
     The 26-year-old Bolt has now collected eight gold medals at world \
    championships, equaling the record held by American trio Carl Lewis, \
    Michael Johnson and Allyson Felix, not to mention the small matter of \
    six Olympic titles.
     The relay triumph followed individual successes in the 100 and 200 \
    meters in the Russian capital.
     \"I'm proud of myself and I'll continue to work to dominate for as \
    long as possible,\" Bolt said, having previously expressed his \
    intention to carry on until the 2016 Rio Olympics.
     Victory was never seriously in doubt once he got the baton safely in \
    hand from Ashmeade, while Gatlin and the United States third leg \
    runner Rakieem Salaam had problems.
     Gatlin strayed out of his lane as he struggled to get full control \
    of their baton and was never able to get on terms with Bolt.
     Earlier, Jamaica's women underlined their dominance in the sprint \
    events by winning the 4x100m relay gold, anchored by Shelly-Ann \
    Fraser-Pryce, who like Bolt was completing a triple.
     Their quartet recorded a championship record of 41.29 seconds, well \
    clear of France, who crossed the line in second place in 42.73 seconds.
     Defending champions, the United States, were initially back in the \
    bronze medal position after losing time on the second handover \
    between Alexandria Anderson and English Gardner, but promoted to \
    silver when France were subsequently disqualified for an illegal \
    handover.
     The British quartet, who were initially fourth, were promoted to the \
    bronze which eluded their men's team.
     Fraser-Pryce, like Bolt aged 26, became the first woman to achieve \
    three golds in the 100-200 and the relay."};

I tried to build a function, but it seems not working properly:

getTextPart[text_]:=Module[{textSentences,numberOfparagraph},
text1=fulltext[[text]];
textSentences=TextSentences[text1];
numberOfparagraph=Length[textSentences]/4//N//Round;
Take[textSentences,numberOfparagraph]]

where full text is the whole documents, containing the two texts; text sentences is the number of sentences in each text of document; number of paragraphs is the number of parts in which we want to split the text (paragraphs).

For example: the text contains 67 sentences, based on the newlines and the dots. We want to split the text in 4 paragraphs.

I would like also know how I can split the paragraphs based on a number of words, instead of sentences.

Thanks for your help.

$\endgroup$
  • $\begingroup$ How can you tell where a new paragraph begins? $\endgroup$ – MassDefect Jul 7 at 5:46
  • $\begingroup$ The number of sentences is counted considering the newlines(indent) and the period. The number of paragraphs is 4 and takes into account the number of sentences of the text (each text). The paragraph should finish with a period. $\endgroup$ – math.world Jul 7 at 5:50
  • $\begingroup$ I'm not exactly sure where you're getting 67 sentences from. TextSentences on each document results in 13, for a total of 26 sentences. Assuming the links at the bottom count as sentences, I get 15 sentences for the first one and 13 for the second. Further, the first document appears to be broken into 12 paragraphs on the CNN website. Does that matter? Or would you prefer to go with your own definition for a paragraph? When you say it should be broken into 4 paragraphs, does it matter which sentences get included in each paragraph? $\endgroup$ – MassDefect Jul 7 at 6:10
  • $\begingroup$ Thanks MassDefect. Mine was just an example. The text that I copied and paste is a sample of what I am using. What would be important is to split the two texts into ac certain number of paragraphs, respectively. To split each of them into, for example, 4 paragraphs, I need to take into account the number of sentences and check that the last sentence is completed, not broken. What do you mean with " does it matter which sentences get included in each paragraph? " $\endgroup$ – math.world Jul 7 at 6:16
  • 1
    $\begingroup$ Okay, I think I'm understanding what the requirements are! I just meant that if you have a document of 17 sentences that you want in 4 paragraphs, how do you decide which sentences go in which paragraphs? Should it be {4, 4, 4, 5}, {5, 4, 4, 4}, or some other combination? $\endgroup$ – MassDefect Jul 7 at 6:39
3
$\begingroup$

Start by splitting the text into sentences as Anton did:

text2 = StringReplace[#, (WhitespaceCharacter ..) -> " "] &@
   StringJoin[{Flatten[text]}];

sentences = TextSentences[text2];

Paragraphs with the same number of sentences

Turn the run of sentences into a graph:

graph = Graph[
  UndirectedEdge @@@ Partition[sentences, 2, 1]
  ]

Mathematica graphics

Use FindGraphPartition to split this graph into four subgraphs approximately equal in size:

paragraphs = FindGraphPartition[graph, 4];
HighlightGraph[graph, Map[Subgraph[graph, #] &, paragraphs]]

Mathematica graphics

Each group has the same number of sentences, approximately:

Length /@ paragraphs

{7, 7, 6, 6}

Paragraphs with the same number of words

FindGraphPartition finds a partition into the specified number of subgraphs such that the size of each subgraph is approximately equal. The size of a subgraph is determined by the sum of its vertex weights. In the previous example, each sentence had the same weight, which means that it tried to find subgraphs that had approximately the same number sentences. To find paragraphs with approximately the same number of words, we specify that the weight of each sentence is its word count.

graph = Graph[
   UndirectedEdge @@@ Partition[sentences, 2, 1],
   VertexWeight -> WordCount /@ sentences
   ];
paragraphs = FindGraphPartition[graph, 4];
HighlightGraph[graph, Map[Subgraph[graph, #] &, paragraphs]]

Mathematica graphics

Length /@ paragraphs

{8, 6, 6, 6}

We see here that the first paragraph has more sentences than the following paragraphs, which, in theory, makes up for the fact that the sentences in the first paragraph are a little bit shorter.

$\endgroup$
  • $\begingroup$ Many thanks, C.E. $\endgroup$ – math.world Jul 7 at 16:12
2
$\begingroup$

For example: the text contains 67 sentences, based on the newlines and the dots. We want to split the text in 4 paragraphs.

The code below splits into 4 paragraphs based on the number of sentences.

I would like also know how I can split the paragraphs based on a number of words, instead of sentences.

A split based on the number of words can be done in a similar manner, but 1) a better definition of what are paragraph split based on the number of words is necessary, and 2) the code, of course, is more complicated.

Code

text2 = StringReplace[#, (WhitespaceCharacter ..) -> " "] &@
   StringJoin[{Flatten[text]}];

sentences = TextSentences[text2];
Length[sentences]

(* 26 *)

n = 4;
paragraphs = Partition[sentences, Floor[Length[sentences]/n]];
If[Length[Flatten[paragraphs]] < Length[sentences],
  paragraphs[[-1]] = 
    Join[paragraphs[[-1]], 
     Take[sentences, -(Length[sentences] - 
         Length[Flatten[paragraphs]])]];
  ];
paragraphs = StringRiffle[#, " "] & /@ paragraphs;

Grid[List /@ paragraphs, Alignment -> Left, Dividers -> All]

enter image description here

$\endgroup$
  • $\begingroup$ Hi Anton Antonov, thank you for your answer. I would like to ask you if I had "n" that is a list containing different numbers (e.g., the number of paragraphs can vary from document to document), how could I change the code? For example: 100 texts and {4,2,3,4...} as n, that means that I need to split the first text in 4 paragraphs, the second one in 2, and so on... $\endgroup$ – math.world Jul 7 at 13:19
  • $\begingroup$ You can use Fold and Take. $\endgroup$ – Anton Antonov Jul 7 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.