0
$\begingroup$

I have 3000 files (for example: aa101.csv, aa108.csv, aa120.csv, ...) and I need to process them. I need to load five files from a folder, process them, and export them to new list in Excel. Then I need to take another five files from the same folder and process them, and so on.

I tried

printer[n_] := 
  Module[{file = n},

    calculation...
    ...

    Export[name <> ".xlsx", {data}, "Data"];
  ]

printer[Import["E:\\mathematica\\a\\*", "text"]];

But printer loads all files from the specified folder.

Could you please help me with this problem?

$\endgroup$
  • $\begingroup$ Function printer take name of the file or data from file? $\endgroup$ – molekyla777 Jul 8 '14 at 12:30
  • $\begingroup$ This function takes data from files $\endgroup$ – Lefl Jul 9 '14 at 7:00
2
$\begingroup$

This somewhat depends on your specific file names, desired target filename and location, the details of procedure etc. There would be a number of approaches. I post this for motivation.

In the following I assume:

  • all the files for processing are in the same directory (and number 0 Mod 5)
  • they have a sequential nomenclature that will be preserved
  • they are .csv
  • the target directory is the same folder

and will use "file001.xlsx","file002.xlsx",...etc and the procedure is a function fun:

SetDirectory["your directory"]
(* get file names and check in desired order*)
fn=FileNames["*.csv"]
(* partition into desired set of 5 *)
pfn=Partition[fn,5];
(* procedure on 5 files *)
proc=Map[Function[x,fun[Import[#,"Text"]&/@x]],pfn]
(*this produces the processed files in a list*)
(*exporting list*)
MapIndexed[Export[
 "file" <> IntegerString[First@#2, 10, 3] <> ".xlsx",#1,"Data"] &, proc]

Whatever approach you ultimately use I suggest testing on small test set to confirm desired outcome.

$\endgroup$
  • $\begingroup$ There is problem with function fun, it doesnt load data from files. Output: Import::chtype: First argument #1 is not a valid file, directory, or URL specification. >> {fun[{$Failed["file001.csv"], $Failed["file002.csv"], $Failed["file003.csv"], $Failed["file004.csv"], $Failed["file005.csv"]}], fun[{$Failed["file006.csv"], $Failed["file007.csv"], $Failed["TS2404_T1-8042.csv"], $Failed["TS2406_T1-806new2.csv"], $Failed[ "TS5620_T1-1484new.csv"]}]} $\endgroup$ – Lefl Jul 9 '14 at 7:11
  • $\begingroup$ @Lefl as I commented fun was just the name I used for the user defined function/procedure hence in this code it is "dummy code" for insert whatever your procedure. I explicitly listed the point of the post was motivation. There was insufficient information in the question for a "solution". $\endgroup$ – ubpdqn Jul 9 '14 at 9:30
  • $\begingroup$ I can´t give my calculations in "fun" because I can´t load data from files. Do you know please what I have to change to load data from files? $\endgroup$ – Lefl Jul 9 '14 at 9:55
  • $\begingroup$ @Lefl if you just wish to get import the data then run your procedure: imported=Map[Function[x,Import[#,"Text"]/@x],pfn] should produce a list, each element will 5 imported csv files. Then you can run your procedure. $\endgroup$ – ubpdqn Jul 9 '14 at 10:09
  • 1
    $\begingroup$ It still doesnt work :( $\endgroup$ – Lefl Jul 9 '14 at 11:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.