3
$\begingroup$

Is it possible to split a large text file into smaller files by line numbers in Mathematica?

$\endgroup$
2
$\begingroup$

Yes, of course.

text = Import["http://www.norvig.com/big.txt"];
StringLength[text]

(* 6488665 *)

lines = StringSplit[text, "\n"];
Length[lines]

(* 128457 *)

breaks = Union[Append[Range[1, Length[lines], 20000], Length[lines]]];

MapIndexed[
 Export["~/Downloads/textPart" <> ToString[#2[[1]]] <> ".txt", 
  Take[lines, #1], "Text"] &, Partition[breaks, 2, 1]]

(* {"~/Downloads/textPart1.txt", "~/Downloads/textPart2.txt", \
"~/Downloads/textPart3.txt", "~/Downloads/textPart4.txt", \
"~/Downloads/textPart5.txt", "~/Downloads/textPart6.txt", \
"~/Downloads/textPart7.txt"}*)

Verification with Unix / Mac OS commands:

[14:40:26]Downloads> ls | grep textPart | xargs -Iaaa wc aaa
   20000  239243 1398793 textPart1.txt
   20000  166040 1042501 textPart2.txt
   20000  155655  948331 textPart3.txt
   20000  166764  948771 textPart4.txt
   20000  169422  964634 textPart5.txt
   20000  167882  957716 textPart6.txt
    8456   30737  228187 textPart7.txt
$\endgroup$
  • $\begingroup$ Thank you very much..!! $\endgroup$ – Bikash Jun 22 '18 at 20:57
  • $\begingroup$ No problem, good luck! $\endgroup$ – Anton Antonov Jun 22 '18 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.