0
$\begingroup$

I'm batch importing some summary files for analysis

SummaryFileList = FileNames["*.csv*"];
AllSummaryFiles = 
Table[
        SummaryFile = 
        Import[SummaryFileList [[i]]],
        {i, 1, NumberOfSummaryFiles}
     ]

Which will produce

{AllSummaryFiles[[1]], AllSummaryFiles[[2]],...,AllSummaryFiles [[i]] }

I'd like to join the output of these tables, at the moment the only way to achieve the desired result is to manualy choose each element of the list of lists as:

Join[AllSummaryFiles[[1]], AllSummaryFiles[[2]],AllSummaryFiles[[3]]]

The problem is obviously the outer brackets of the table output is there a way to acheive the above result without manually picking out the induvidual list.


Essentially, if I import all my summary files sequentially I want to join them all together so that they form a big long list in the order in which they were imported.

$\endgroup$

1 Answer 1

2
$\begingroup$

Try with:

Join @@ Import["*.csv"]

also, are you sure you want the * after csv?

Some explanation of the code above:

Import["*.csv"]

imports all the files above as a list: if you have three .csv files (1.csv,2.csv,3.csv), the output will be

{file_data1,file_data2,file_data3}

This would be equivalent of your code:

SummaryFileList = FileNames["*.csv"];
NumberOfSummaryFiles = Length[SummaryFileList];
AllSummaryFiles = Table[
   Import[SummaryFileList [[i]]],
   {i, 1, NumberOfSummaryFiles}
]

which, however, would read better as:

SummaryFileList = FileNames["*.csv"];
AllSummaryFiles = Table[
   Import[i],
   {i, 1, SummaryFileList}
]

In both cases, the Table loop is redundant as Import deals with many files automatically, hence one should use just Import["*.csv"]

You then want to Join your files: the way to do it is:

Join @@ AllSummaryFiles

which replicates you example:

Join[AllSummaryFiles[[1]], AllSummaryFiles[[2]],AllSummaryFiles[[3]]]

for a generic length of the list AllSummaryFiles

As obvious, it's much more convenient (and also efficient) to type only Join @@ Import["*.csv"] rather than the whole Table loop, unless you have a specific reason for doing so.

$\endgroup$
13
  • $\begingroup$ Your solution just replicates my problem with joining the output of the table I'm afraid. The second * doesn't seem to make a difference for my particular case. $\endgroup$
    – user27119
    Commented Sep 9, 2019 at 13:34
  • $\begingroup$ Could you provide a sample of the data files (a couple of them should be enough), and elaborate on what your problem is? Maybe it's just me, but I don't get what's the problem you need to solve :) $\endgroup$
    – Fraccalo
    Commented Sep 9, 2019 at 13:43
  • 1
    $\begingroup$ Also, I don't think that what you say is true: my code is not functionally the same as Join[{File1, File2, File3}], while it's actually doing Join[File1,File2,File3]: this is because I'm using Apply (@@): for example, f @@ {a, b, c, d} corresponds to f[a, b, c, d], if you replace f with Join it's exactly what (as far as I understand) you are asking. $\endgroup$
    – Fraccalo
    Commented Sep 9, 2019 at 16:00
  • 1
    $\begingroup$ That's not what I meant with my code! You should use my code INSTEAD of your code and it should do what you are asking for. Try it, while I amend my answer implementing a Table version of it (even if it's much more verbose) $\endgroup$
    – Fraccalo
    Commented Sep 9, 2019 at 16:23
  • 1
    $\begingroup$ I'm afraid it was me who was being stupid! $\endgroup$
    – user27119
    Commented Sep 9, 2019 at 16:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.