0
$\begingroup$

I would like to import three files in Mathematica (I am using the Import function[]) and try to clean them. I used

Import[#,"Table"]&/@FileNames["/DirectoryPath/*.txt"]

to import the files (it is a list, where each file is a string of this list). What I would like to know is how to clean each file; for example, if I want to use

StringReplace[StringDelete[data, "@"],"\n"..->"\n"]

to clean all of them, how could I do? At the end, I would like to train a neural network.

Many thanks for your help.

$\endgroup$
  • $\begingroup$ If you store all the files with your import command: data=Import[#...]&/@FileNames[...], then you can do work on each file with cleanedata=StringReplace[StringDelete[#, "@"],"\n"..->"\n"]&/@data $\endgroup$ – MelaGo Jul 6 at 1:42
  • $\begingroup$ Thank you MelaGo. The problem is that I need to use all that data as input in a neural network, so probably I need to run them in parallel (are at least 100 files). I also got the following error: String or list of strings expected at position 1 in StringDelete I was thinking of MapThread but I have never used it. $\endgroup$ – math.world Jul 6 at 1:47
  • $\begingroup$ When you import with "Table" it turns your files into lists. If you want to use string functions like StringReplace with newlines and such, I think you'll have to keep your files in string or text format. Can you post an example file? $\endgroup$ – MelaGo Jul 6 at 2:00
  • $\begingroup$ An example of files that I am importing is: history1.txt, history2.txt,..., history100.txt Each file contains many lines of text, for example : "The 4th of July in the United States....". $\endgroup$ – math.world Jul 6 at 2:40
1
$\begingroup$

For illustration, here are three text files (copied from the first three paragraphs of this Wikipedia page), called "history1.txt", "history2.txt", and "history3.txt".

data = Import[#] & /@ FileNames["history*.txt"]
(*
{"Independence Day (colloquial: the Fourth of July) is a federal \
holiday in the United States commemorating the Declaration of \
Independence of the United States, on July 4, 1776. ...", \
"Independence Day is commonly associated with fireworks, parades, \
barbecues, carnivals, fairs, picnics, concerts, baseball games, \
family reunions, political speeches, and ceremonies, ...", 
"During the American Revolution, the legal separation of the Thirteen \
Colonies from Great Britain in 1776 actually occurred on July 2, when \
the Second Continental Congress voted to approve a resolution of \
independence ..."}
*)

You can then do some work on all of the files. For example:

StringReplace[#, "Independence Day" -> "July 4th"] & /@ data

(*
{"July 4th (colloquial: the Fourth of July) is a federal holiday in \
the United States commemorating the Declaration of Independence of \
the United States, on July 4, 1776. ...", 
"July 4th is commonly \
associated with fireworks, parades, barbecues, carnivals, fairs, \
picnics, concerts, baseball games, family reunions, political \
speeches, and ceremonies, ...", 
"During the American Revolution, the \
legal separation of the Thirteen Colonies from Great Britain in 1776 \
actually occurred on July 2, when the Second Continental Congress \
voted to approve a resolution of independence ..."}
*)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.