2
$\begingroup$

I am trying to calculate the eigenvalues of a large matrix (10^4x10^4), which is formed by summing a matrix containing advective terms and a matrix containing diffusive terms. I am using the following simple script:

add = "address/n=50l=50/";
adv = Transpose[Import[add <> "u_B.dat", "Table"]];

diff = Transpose[Import[add<>"diffusion.dat", "Table"]];

ans = 303*adv + diff // Eigenvalues;
Sort[ans]

Mathematica imports the diffusion matrix as a one dimensional array for some reason, and as such cannot transpose it or calculate eigenvalues. I have used this code for various (smaller) resolutions/matrix sizes and it worked fine. I really do not understand why this matrix is suddenly being imported as a one dimensional object.

----------Edit 2---------

Here is a link to the diffusion file: https://www.dropbox.com/s/gx8sokt2zggiq99/diffusion.dat?dl=0


Here are the first few elements of the first row. There are 10^4 rows and 10^4 columns in the file (adv has the same)

  -9.9999999999997868      -2.0718235518950072      -0.7368652300812841      -0.3366733737341420      -0.1785767012502300      -0.1046968732429871      -0.0659813101416420      -0.0439219818289296      -0.0305217022632576      -0.0219581437532483      -0.0162554151475518      -0.0123258483194475      -0.0095389926277596      -0.0075132770178053      -0.0060091173707776      -0.0048712446774423      -0.0039962212568781      -0.0033134493535647      -0.0027736997195499      -0.0023419799843653      -0.0019929858760674      -0.0017081324330329      -0.0014735758787324      -0.0012788706178710      -0.0011160417221390      -0.0009789342582586      -0.0008627502850535      -0.0007637149330716      -0.0006788326779994      -0.0006057073883789      -0.0005424080705597      -0.0004873676971277      -0.0004393062644368      -0.0003971718205800      -0.0003600948026407      -0.0003273525133293      -0.0002983412085359      -0.0002725540935550      -0.0002495637478567      -0.0002290081574144      -0.0002105794044882      -0.0001940145717756      -0.0001790883002599      -0.0001656067401675      -0.0001534025872206      -0.0001423309440523      -0.0001322659628047      -0.0001230979832230      -0.0001147311994253      -0.0001070816570830       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000       0.0000000000000000
$\endgroup$
8
  • $\begingroup$ Can you paste first few columns and rows from your file? Maybe it is not what you think it is. $\endgroup$
    – Kuba
    Commented Feb 5, 2019 at 10:29
  • $\begingroup$ I should add, if I display a part of the diff table it looks fine {{},{},....{}}, just like the adv table, but when I ask Mathematica what its dimensions are it gives me only 1 dimension (10^4).. $\endgroup$
    – Rossana
    Commented Feb 5, 2019 at 10:45
  • $\begingroup$ To help debug this problem it is important to know the Dimensions[] of the matrices immediately after the Import[] but before the Transpose[]. Could you check if it is what you think it should be. Also the arrays you import, are they square? Eigenvalues[] by default requires a square matrix. I don't see how 303*adv + diff is square unless both adv and diff are also square. $\endgroup$
    – Somos
    Commented Feb 5, 2019 at 12:09
  • $\begingroup$ After importing the dimensions are {10'000,10'000} for adv (irrespective of transposition) and {10'000} for diff, which is odd, because when I print it it looks like it should have the same dimension as adv. Sorry I made a mistake earlier, the file does not have 250000 elements, that's just the number of characters. $\endgroup$
    – Rossana
    Commented Feb 5, 2019 at 12:20
  • $\begingroup$ It's going to be nearly impossible to diagnose this without the file - it could be as simple as two numbers running together and freaking out the interpreter or one row with a different number of entries. If you could upload it somewhere someone can probably help. If you don't want to do that, I would try using OpenRead and ReadLine and then StringSplit and ToString to get your data in the proper form - it will be a little more work, but it will make it easier to figure out where things go wrong. $\endgroup$
    – N.J.Evans
    Commented Feb 5, 2019 at 14:19

1 Answer 1

5
$\begingroup$

I can't tell you why this is occurring, but there are 4 lines which are each short by one number. You can see the line numbers below: 9850, 9900, 9950, 1000.

diffusion = Import["diffusion.dat", "Table"];

(* save the import in case its needed later *)
DumpSave["diffusion.mx", diffusion];

(* is it nxn? *)
Dimensions@diffusion

(* {10000} *)

(* no, what lengths are present? *)
Length /@ diffusion // Union

(* {9999,10000} *)

(* where are the short lines? *)
lengths = Table[{n, Length[diffusion[[n]]]}, {n, Length[diffusion]}];
Select[lengths, #[[2]] == 9999 &]

(* {{9850,9999},{9900,9999},{9950,9999},{10000,9999}} *)

Edit: Further, this can be used to examine line 9850 as a string:

stream = OpenRead["diffusion.dat"];

Skip[stream, "String", 9849];

rec9850 = ReadList[stream, "String", 1]

In a section of that string you will see a run-together number:

enter image description here

Sorry, another edit, but this problem is too interesting. Assuming this run-together at the minus sign is the issue, it can be corrected by putting a space in front of all the minus signs:

SetStreamPosition[stream, 0];

strings = ReadList[stream, "String"];

correctedStrings = StringReplace[#, "-" -> " -"] & /@ strings;

diffusion2 = 
  ReadList[StringToStream[#], "Number"] & /@ correctedStrings;

diffusion2 // Dimensions

(* {10000,10000} *)

And:

eigenValues = Eigenvalues[diffusion2];

Histogram[eigenValues, PlotTheme -> "Scientific"]

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.