0
$\begingroup$

I would like to understand what I am doing wrong in this function:

    $maxnumchar = 5000;

processSample[text_, numcaption, caption_] := Module[{parts, story, sentences, paragraphs, puncts, maxpos},
    parts = StringSplit[text,"@caption"];
    story = StringReplace[First@parts, "\n\n"->"\n"];
    sentences=TextSentences[story];
    paragraphs=Partition[sentences, Floor[Length[sentences]/Length[numcaption]]];
    If[StringLength[story]>$maxnumchar,
    If[Length[Flatten[paragraphs]]<Length[sentences],
        paragraphs[[-1]]= Join[paragraphs[[-1]],
                            Take[sentences,-(Length[sentences]-Length[Flatten[paragraphs]])]
                              ];
                    ];
        paragraphs=StringRiffle[#," "]&/@paragraphs;
    ];
    story -> parts[[2]];
    Table[paragraphs[[i]]->caption[[i]],{i,numcaption}]
];

I give you some context: I posted a question where I asked how to split the text into paragraphs. And I received two good answers, because both working well. Now, what I am trying to do is to write a function that clean the text from empty spaces between lines and remove the word "@caption" from the text. I would like to split the paragraphs taking into the account the number of captions in the text; but it also true that I would need a maximum of 5000 characters per paragraph. I would like to build a function the could give me the opportunity to choose between the maximum of characters or the number of captions in the text. For splitting the text into paragraphs, I am considering once again the number of sentences in the text. A sample of texts is:

text = {"It's official: U.S. President Barack Obama wants lawmakers to weigh \
    in on whether to use military force in Syria.
     Obama sent a letter to the heads of the House and Senate on Saturday \
    night, hours after announcing that he believes military action \
    against Syrian targets is the right step to take over the alleged use \
    of chemical weapons.
     The proposed legislation from Obama asks Congress to approve the use \
    of military force \"to deter, disrupt, prevent and degrade the \
    potential for future uses of chemical weapons or other weapons of \
    mass destruction.\"
     It's a step that is set to turn an international crisis into a \
    fierce domestic political battle.
     There are key questions looming over the debate: What did U.N. \
    weapons inspectors find in Syria? What happens if Congress votes no? \
    And how will the Syrian government react?
     In a televised address from the White House Rose Garden earlier \
    Saturday, the president said he would take his case to Congress, not \
    because he has to -- but because he wants to.
     \"While I believe I have the authority to carry out this military \
    action without specific congressional authorization, I know that the \
    country will be stronger if we take this course, and our actions will \
    be even more effective,\" he said. \"We should have this debate, \
    because the issues are too big for business as usual.\"
     Obama said top congressional leaders had agreed to schedule a debate \
    when the body returns to Washington on September 9. The Senate \
    Foreign Relations Committee will hold a hearing over the matter on \
    Tuesday, Sen. Robert Menendez said.
     Transcript: Read Obama's full remarks
     Syrian crisis: Latest developments
     U.N. inspectors leave Syria
     Obama's remarks came shortly after U.N. inspectors left Syria, \
    carrying evidence that will determine whether chemical weapons were \
    used in an attack early last week in a Damascus suburb.




     ", "(CNN) -- Usain Bolt rounded off the world championships Sunday \
    by claiming his third gold in Moscow as he anchored Jamaica to \
    victory in the men's 4x100m relay.
     The fastest man in the world charged clear of United States rival \
    Justin Gatlin as the Jamaican quartet of Nesta Carter, Kemar \
    Bailey-Cole, Nickel Ashmeade and Bolt won in 37.36 seconds.
     The U.S finished second in 37.56 seconds with Canada taking the \
    bronze after Britain were disqualified for a faulty handover.
     The 26-year-old Bolt has now collected eight gold medals at world \
    championships, equaling the record held by American trio Carl Lewis, \
    Michael Johnson and Allyson Felix, not to mention the small matter of \
    six Olympic titles.
     The relay triumph followed individual successes in the 100 and 200 \
    meters in the Russian capital.
     \"I'm proud of myself and I'll continue to work to dominate for as \
    long as possible,\" Bolt said, having previously expressed his \
    intention to carry on until the 2016 Rio Olympics.
     Victory was never seriously in doubt once he got the baton safely in \
    hand from Ashmeade, while Gatlin and the United States third leg \
    runner Rakieem Salaam had problems.
     Gatlin strayed out of his lane as he struggled to get full control \
    of their baton and was never able to get on terms with Bolt.
     Earlier, Jamaica's women underlined their dominance in the sprint \
    events by winning the 4x100m relay gold, anchored by Shelly-Ann \
    Fraser-Pryce, who like Bolt was completing a triple.
     Their quartet recorded a championship record of 41.29 seconds, well \
    clear of France, who crossed the line in second place in 42.73 seconds.
     Defending champions, the United States, were initially back in the \
    bronze medal position after losing time on the second handover \
    between Alexandria Anderson and English Gardner, but promoted to \
    silver when France were subsequently disqualified for an illegal \
    handover.
     The British quartet, who were initially fourth, were promoted to the \
    bronze which eluded their men's team.
     Fraser-Pryce, like Bolt aged 26, became the first woman to achieve \
    three golds in the 100-200 and the relay."};

where full text is the whole documents, containing the two texts; text sentences is the number of sentences in each text of document; number of paragraphs is the number of parts in which I want to split the text (paragraphs). What I would like is also to do some association between paragraphs and captions. For example: paragraph[1]->caption[1], paragraph[2]->caption[2] (in each text). The number of captions changes from text to text as well as the number of paragraphs because of the number of characters/words used. I would like to try considering the content, in order to do an association based on similarity, but I think it would be too complex.

I hope you can help me.

Many thanks.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.