3
$\begingroup$

I have a large text file. I read data from text file one at a time and check whether data agree with my condition. If not I want to delete that data in text file and update it.

For example say my data text file is: 100 201 302 455

Suppose I want to delete second one. After deletion I want new text file to be look like 100 302 455 (without any gap between 100 and 302). Since my text file is large around 100MB, I want to read data one by one rather than loading all at once. Thanks

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Just use sed and be done with it :) something like sed --in-place '/201/d' file.txt $\endgroup$
    – Nasser
    Sep 1, 2013 at 22:46
  • $\begingroup$ @Nasser Surely dedicated text utilities are a better approach for simple replacements. I have to assume that the condition here is nontrivial and requires or at least benefits from Mathematica processing. $\endgroup$
    – Mr.Wizard
    Sep 1, 2013 at 22:50
  • $\begingroup$ @Mr.Wizard yes ofcourse, I understand that. I was just giving an answer for the specific example (I also put a smiley, there, just in case :) $\endgroup$
    – Nasser
    Sep 1, 2013 at 23:07
  • $\begingroup$ @Nasser From your response I think I failed to communicate what I intended. Let me try again: Using an external text processing utility is a great idea and should be used whenever possible as it will deliver superior performance in nearly all cases. $\endgroup$
    – Mr.Wizard
    Sep 1, 2013 at 23:10

1 Answer 1

Reset to default
7
$\begingroup$

100MB is small compared to today's RAM sizes. Why not load it all at once? Then it's just a matter of using DeleteCases and exporting the file.

Otherwise I don't believe Mathematica is natively equipped to modify a file piece by piece in that fashion so you'll need to export to a second file. As an example I'll filter a list of natural numbers to keep only the primes. (It would be a trivial use of Not to drop the primes.)

First generate the starting data:

Export["firstfile.txt", Range@100, "Table"];

Open the input and output streams:

in = OpenRead["firstfile.txt"];
out = OpenWrite["secondfile.txt"];

Read, filter, and export:

Module[{x},
 While[x =!= EndOfFile,
  x = Read[in];
  If[PrimeQ@x, Write[out, x]]
 ]
]

Close the streams:

Scan[Close, {in, out}]

The result:

FilePrint["secondfile.txt"]

2
3
5
7
11
13
...

Note: practically I would include in and out in my Module but here it made it harder to comment the code as I wanted.

Improve this answer
$\endgroup$
2
  • $\begingroup$ thank you for the answer. It really helped me. Thank you again for the help. $\endgroup$
    – Vajira
    Sep 2, 2013 at 18:04
  • $\begingroup$ @Vajira You're welcome. If you find the answer fully satisfactory please consider Accepting it. $\endgroup$
    – Mr.Wizard
    Sep 2, 2013 at 23:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.