# Fast way to add a column to a large matrix

I have used the (apparently fastest) approach of

Do[MapThread[Prepend, {m, v}];


for prepending and appending v to a matrix m from question What is the most efficient way to add rows and columns to a matrix? but my matrix is quite large and it is too slow for my purposes. (Not at home now but rough performance guide is ~30 seconds to append a 5,000 row vector to a 5000 row matrix to 2 cols)

The specific context: a matrix of time series data (imported from CSV) of 50,000 rows (could be MUCH longer) and 2 columns (col 1 timestamp; col 2 value). I want to add a third column and re-export to CSV.

Is there a better approach than the method quoted?

• mathematica.stackexchange.com/a/3077/21532
– yode
Mar 24, 2017 at 7:11
• It is unclear from your description: do you actually use the imported data to create the new column? If not, there seems no need to even create the matrix. And even if you do need to actually import the data into a matrix, you need not create the bigger matrix to export the data.
– Alan
Mar 24, 2017 at 14:17
• Is the matrix packed? Mar 24, 2017 at 17:04
• @J.M.No, not packed. Mar 25, 2017 at 9:35

## 2 Answers

Adding the column shouldn't be too time consuming. Here's a 1,000,000 x 2 matrix of real numbers getting a column added, with two Transpose[ ] calls thrown in. Too cheap to meter. The file writing is the expensive part.

In[107]:= n = 1000000;

In[108]:= res = Table[RandomReal[20], {i, n}, {j, 2}];

In[109]:= vec = Table[RandomReal[20], {i, n}];

In[110]:= Timing[tres = Append[Transpose[res], vec] // Transpose;]

Out[110]= {0., Null}

In[111]:= Timing[Export["test.csv", tres]]

Out[111]= {19.2193, "test.csv"}

• (Working to a deadline so no time to got back and investigate but to acknowledge and thank...) Yes, that seems fast enough ;) I think the problem may have been that one column was of reals and the other of DateObjects. I've bypassed the conversion of timestamps to DateObjects for now and with only text/numbers it is plenty fast. Thx again Mar 25, 2017 at 9:41
mat = RandomInteger[100, {10, 2}];  (* fake matrix *)
src = "/temp/temp99.csv";  (* location for fake data *)
Export[src, mat];
vec = RandomInteger[100, 10];  (* fake vector *)
tmp = StringTemplate[","];  (* string template for conversion *)
fOld = OpenRead[src]; (* open your data file to read *)
fNew = OpenWrite["/temp/temp98.csv"] (* target for new data *)
(* each item of vec, read old line & write new line *)
Scan[WriteLine[fNew, tmp[ReadLine[fOld], #]] &, vec];
Close[fNew]; Close[fOld];