I would like to understand what I am doing wrong in this function:

    $maxnumchar = 5000;

processSample[text_, numcaption, caption_] := Module[{parts, story, sentences, paragraphs, puncts, maxpos},
    parts = StringSplit[text,"@caption"];
    story = StringReplace[First@parts, "\n\n"->"\n"];
    paragraphs=Partition[sentences, Floor[Length[sentences]/Length[numcaption]]];
        paragraphs[[-1]]= Join[paragraphs[[-1]],
        paragraphs=StringRiffle[#," "]&/@paragraphs;
    story -> parts[[2]];

I give you some context: I posted a question where I asked how to split the text into paragraphs. And I received two good answers, because both working well. Now, what I am trying to do is to write a function that clean the text from empty spaces between lines and remove the word "@caption" from the text. I would like to split the paragraphs taking into the account the number of captions in the text; but it also true that I would need a maximum of 5000 characters per paragraph. I would like to build a function the could give me the opportunity to choose between the maximum of characters or the number of captions in the text. For splitting the text into paragraphs, I am considering once again the number of sentences in the text. A sample of texts is:

text = {"It's official: U.S. President Barack Obama wants lawmakers to weigh \
    in on whether to use military force in Syria.
     Obama sent a letter to the heads of the House and Senate on Saturday \
    night, hours after announcing that he believes military action \
    against Syrian targets is the right step to take over the alleged use \
    of chemical weapons.
     The proposed legislation from Obama asks Congress to approve the use \
    of military force \"to deter, disrupt, prevent and degrade the \
    potential for future uses of chemical weapons or other weapons of \
    mass destruction.\"
     It's a step that is set to turn an international crisis into a \
    fierce domestic political battle.
     There are key questions looming over the debate: What did U.N. \
    weapons inspectors find in Syria? What happens if Congress votes no? \
    And how will the Syrian government react?
     In a televised address from the White House Rose Garden earlier \
    Saturday, the president said he would take his case to Congress, not \
    because he has to -- but because he wants to.
     \"While I believe I have the authority to carry out this military \
    action without specific congressional authorization, I know that the \
    country will be stronger if we take this course, and our actions will \
    be even more effective,\" he said. \"We should have this debate, \
    because the issues are too big for business as usual.\"
     Obama said top congressional leaders had agreed to schedule a debate \
    when the body returns to Washington on September 9. The Senate \
    Foreign Relations Committee will hold a hearing over the matter on \
    Tuesday, Sen. Robert Menendez said.
     Transcript: Read Obama's full remarks
     Syrian crisis: Latest developments
     U.N. inspectors leave Syria
     Obama's remarks came shortly after U.N. inspectors left Syria, \
    carrying evidence that will determine whether chemical weapons were \
    used in an attack early last week in a Damascus suburb.

     ", "(CNN) -- Usain Bolt rounded off the world championships Sunday \
    by claiming his third gold in Moscow as he anchored Jamaica to \
    victory in the men's 4x100m relay.
     The fastest man in the world charged clear of United States rival \
    Justin Gatlin as the Jamaican quartet of Nesta Carter, Kemar \
    Bailey-Cole, Nickel Ashmeade and Bolt won in 37.36 seconds.
     The U.S finished second in 37.56 seconds with Canada taking the \
    bronze after Britain were disqualified for a faulty handover.
     The 26-year-old Bolt has now collected eight gold medals at world \
    championships, equaling the record held by American trio Carl Lewis, \
    Michael Johnson and Allyson Felix, not to mention the small matter of \
    six Olympic titles.
     The relay triumph followed individual successes in the 100 and 200 \
    meters in the Russian capital.
     \"I'm proud of myself and I'll continue to work to dominate for as \
    long as possible,\" Bolt said, having previously expressed his \
    intention to carry on until the 2016 Rio Olympics.
     Victory was never seriously in doubt once he got the baton safely in \
    hand from Ashmeade, while Gatlin and the United States third leg \
    runner Rakieem Salaam had problems.
     Gatlin strayed out of his lane as he struggled to get full control \
    of their baton and was never able to get on terms with Bolt.
     Earlier, Jamaica's women underlined their dominance in the sprint \
    events by winning the 4x100m relay gold, anchored by Shelly-Ann \
    Fraser-Pryce, who like Bolt was completing a triple.
     Their quartet recorded a championship record of 41.29 seconds, well \
    clear of France, who crossed the line in second place in 42.73 seconds.
     Defending champions, the United States, were initially back in the \
    bronze medal position after losing time on the second handover \
    between Alexandria Anderson and English Gardner, but promoted to \
    silver when France were subsequently disqualified for an illegal \
     The British quartet, who were initially fourth, were promoted to the \
    bronze which eluded their men's team.
     Fraser-Pryce, like Bolt aged 26, became the first woman to achieve \
    three golds in the 100-200 and the relay."};

where full text is the whole documents, containing the two texts; text sentences is the number of sentences in each text of document; number of paragraphs is the number of parts in which I want to split the text (paragraphs). What I would like is also to do some association between paragraphs and captions. For example: paragraph[1]->caption[1], paragraph[2]->caption[2] (in each text). The number of captions changes from text to text as well as the number of paragraphs because of the number of characters/words used. I would like to try considering the content, in order to do an association based on similarity, but I think it would be too complex.

I hope you can help me.

Many thanks.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.