# Importing data into an array, column by column

I'm building a large matrix having different sized rows consisting of a few columns. The different columns are imported .txt files.

Why can't I build the matrix column by column? For example, I thought I could populate the first column by evaluating

NumberConcentration [[All, 1]] =
Import["C:\\Users...NumberConcentration.txt", "Table"][[87]]


Mathematica gives me an error saying this part of the array does not exist.

Set::partd: Part specification NumberConcentration[[All, 1]] is longer than depth of object. >>

Do I have to define intermediate values and then combine all of them to one matrix? Or is my syntax wrong?

-

To make your syntax work, you can define

NumberConcentration = ConstantArray[{},100];


Then use

NumberConcentration[[1]] = Import["C:\\Users...NumberConcentration.txt", "Table"][[87]]


where 100 should be replaced by the number of files you want to import. This should get rid of the error message. You don't need to have the All there.

Or, you can bypass this step by making a list of all the file names (here files) and then importing them all at once

files = {"file1.txt", "file2.txt"};
NumberConcentration = Import[#, "Table"][[87]] & /@ files


If you want to attach a path to all the files (as asked in the comments):

path = "C\\:this\\that\\";;
fullFiles = First[Outer[StringJoin, {path}, files]]

{"C:\this\that\file1.txt", "C:\this\that\file2.txt"}


which builds the list of full-path file names.

-
related to this: do you know if I can shorten the filepath string. For example, if my Mathematica script is in C:\ProgramFiles, and I want to refer to a file in a folder inside ProgramFiles folder, can I do that in mathematica with a shorter reference string than C:\ProgramFiles\Folder\File.txt. For example, Import["Folder\File"], or something like that –  l3win Jul 6 '13 at 8:01
Say you have path="C:\this\that\". Then a construct like Table[Import[path<>files[[i]]],{i,1,Length[files]}] should work. (<> is short for StringJoin.) You may have to "escape" the backslashes, i.e., use double backslashes. –  bill s Jul 6 '13 at 8:05
@l3win consider adding this comment as an edit to the question. –  Kuba Jul 6 '13 at 8:50

For example, if my Mathematica script is in C:\ProgramFiles, and I want to refer to a file in a folder inside ProgramFiles folder, can I do that in mathematica with a shorter reference string than C:\ProgramFiles\Folder\File.txt [...] - l3win

Automatization

SetDirectory@NotebookDirectory[];
files = FileNames["*", {FileNameJoin[{Directory[], "FolderName"}]}];


Importing 1

Next, assuming You want to Import all files from FolderName, bill s's way is natural also to me. But, You have also said that those should be collumns. Assuming each has equal length we need to do transposition:

 NumberConcentration = Import[#, "Table"][[87]] & /@ files;
NumberConcentration = Transpose@NumberConcentration;


Importing 2

But Your way is also possible. You only have to define earlier what NumberConcentration is. Column length - 100 can be set by import first file and checking Length of it.

NumberConcentration = ConstantArray[ ,{100, Length@files}];
Do[NumberConcentration[[;;, i]] = Import[files[[i]], "Table"][[87]], {i,Length@files}];


Summary

Big advantage is that You only need to type "FolderName" once, and no file names manually. If there are also other files in FolderName, and You only want to Import those of txt type, then simply FileNames["*.txt", ...

There might be some typos in code, I can not check if it is working now.

-
Heya Kuba, I +1ed. Just a quick note, expr[[;; , i]] is the same as expr[[All,i]], right? As the OP also used All. –  Jacob Akkerboom Jul 6 '13 at 9:34
Heya @JacobAkkerboom :) Yes it is the same. Thanks for +1, do You find this useful? :) –  Kuba Jul 6 '13 at 9:37
yes, I like that you show that you can also use Mathematicas "directory functions": FileNameJoin, Directory, SetDirectory and NotebookDirectory. –  Jacob Akkerboom Jul 6 '13 at 9:54
@JacobAkkerboom I think it is a good habit to use them if You have more than one table to import. It is so convenient and fast. –  Kuba Jul 6 '13 at 9:57
Thanks for all these comments. I have one follow up, related to this: NumberConcentration = Import[#, "Table"][[87]] & /@ files; I'm taking all the values in row 87 for all the files in "files." However, my data of interest, depending on the file, is in row 87 or 88. I'm not sure how to incorporate that into the code since I'm not really familiar with how the # and @/ operators work –  l3win Jul 7 '13 at 0:48