# Mathematica incorrectly imports a table

I am trying to calculate the eigenvalues of a large matrix (10^4x10^4), which is formed by summing a matrix containing advective terms and a matrix containing diffusive terms. I am using the following simple script:

add = "address/n=50l=50/";

diff = Transpose[Import[add<>"diffusion.dat", "Table"]];

ans = 303*adv + diff // Eigenvalues;
Sort[ans]


Mathematica imports the diffusion matrix as a one dimensional array for some reason, and as such cannot transpose it or calculate eigenvalues. I have used this code for various (smaller) resolutions/matrix sizes and it worked fine. I really do not understand why this matrix is suddenly being imported as a one dimensional object.

----------Edit 2---------

Here is a link to the diffusion file: https://www.dropbox.com/s/gx8sokt2zggiq99/diffusion.dat?dl=0

Here are the first few elements of the first row. There are 10^4 rows and 10^4 columns in the file (adv has the same)

  -9.9999999999997868      -2.0718235518950072      -0.7368652300812841      -0.3366733737341420      -0.1785767012502300      -0.1046968732429871      -0.0659813101416420      -0.0439219818289296      -0.0305217022632576      -0.0219581437532483      -0.0162554151475518      -0.0123258483194475      -0.0095389926277596      -0.0075132770178053      -0.0060091173707776      -0.0048712446774423      -0.0039962212568781      -0.0033134493535647      -0.0027736997195499      -0.0023419799843653      -0.0019929858760674      -0.0017081324330329      -0.0014735758787324      -0.0012788706178710      -0.0011160417221390      -0.0009789342582586      -0.0008627502850535      -0.0007637149330716      -0.0006788326779994      -0.0006057073883789      -0.0005424080705597      -0.0004873676971277      -0.0004393062644368      -0.0003971718205800      -0.0003600948026407      -0.0003273525133293      -0.0002983412085359      -0.0002725540935550      -0.0002495637478567      -0.0002290081574144      -0.0002105794044882      -0.0001940145717756      -0.0001790883002599      -0.0001656067401675      -0.0001534025872206      -0.0001423309440523      -0.0001322659628047      -0.0001230979832230      -0.0001147311994253      -0.0001070816570830       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000

• Can you paste first few columns and rows from your file? Maybe it is not what you think it is. – Kuba Feb 5 '19 at 10:29
• I should add, if I display a part of the diff table it looks fine {{},{},....{}}, just like the adv table, but when I ask Mathematica what its dimensions are it gives me only 1 dimension (10^4).. – Rossana Feb 5 '19 at 10:45
• To help debug this problem it is important to know the Dimensions[] of the matrices immediately after the Import[] but before the Transpose[]. Could you check if it is what you think it should be. Also the arrays you import, are they square? Eigenvalues[] by default requires a square matrix. I don't see how 303*adv + diff  is square unless both adv and diff are also square. – Somos Feb 5 '19 at 12:09
• After importing the dimensions are {10'000,10'000} for adv (irrespective of transposition) and {10'000} for diff, which is odd, because when I print it it looks like it should have the same dimension as adv. Sorry I made a mistake earlier, the file does not have 250000 elements, that's just the number of characters. – Rossana Feb 5 '19 at 12:20
• It's going to be nearly impossible to diagnose this without the file - it could be as simple as two numbers running together and freaking out the interpreter or one row with a different number of entries. If you could upload it somewhere someone can probably help. If you don't want to do that, I would try using OpenRead and ReadLine and then StringSplit and ToString to get your data in the proper form - it will be a little more work, but it will make it easier to figure out where things go wrong. – N.J.Evans Feb 5 '19 at 14:19

I can't tell you why this is occurring, but there are 4 lines which are each short by one number. You can see the line numbers below: 9850, 9900, 9950, 1000.

diffusion = Import["diffusion.dat", "Table"];

(* save the import in case its needed later *)
DumpSave["diffusion.mx", diffusion];

(* is it nxn? *)
Dimensions@diffusion

(* {10000} *)

(* no, what lengths are present? *)
Length /@ diffusion // Union

(* {9999,10000} *)

(* where are the short lines? *)
lengths = Table[{n, Length[diffusion[[n]]]}, {n, Length[diffusion]}];
Select[lengths, #[[2]] == 9999 &]

(* {{9850,9999},{9900,9999},{9950,9999},{10000,9999}} *)


Edit: Further, this can be used to examine line 9850 as a string:

stream = OpenRead["diffusion.dat"];

Skip[stream, "String", 9849];

rec9850 = ReadList[stream, "String", 1]


In a section of that string you will see a run-together number:

Sorry, another edit, but this problem is too interesting. Assuming this run-together at the minus sign is the issue, it can be corrected by putting a space in front of all the minus signs:

SetStreamPosition[stream, 0];

strings = ReadList[stream, "String"];

correctedStrings = StringReplace[#, "-" -> " -"] & /@ strings;

diffusion2 =
ReadList[StringToStream[#], "Number"] & /@ correctedStrings;

diffusion2 // Dimensions

(* {10000,10000} *)


And:

eigenValues = Eigenvalues[diffusion2];

Histogram[eigenValues, PlotTheme -> "Scientific"]