# Formatting an imported data into Mathematica in a matrix form

I have a file whose data read as {{0,:,1,2,3;},{1,:,0,164,165;},{2,:,0,3,167;},{3,:,0,2,166;},.., {"$$"}} I have to format this data into a matrix. There is also a first three line that I omitted for clarity. I was trying to remove the these lines by hand, but I found it cumbersome for each file. So, the reading of the data should be from the fourth line onward. In this above data {{0,:,1,2,3;},.., {"$$"}}, the first entry tells us the lattice site, and after semi-colon :, it shows to which this lattice site is connected. It is one-dimensional data. Moreover, we should ignore the last part (or last line) in the data, i.e. {"$$"}. Essentially, the first-three and the last lines are to be ignored while reading the data. The truncated data can be found here pastebin. For instance, in this entry {{0,:,1,2,3;}, the lattice site 0 is connected to the lattice site 1, 2, and 3, and so on and so forth. Hence the matrix entry coming from this case would look like {{0,1,1,1,...,0},...} The total lattice sites in the above case is 165, so the matrix would be 165 $$\times$$ 165 My MWE: data=Import["file.dat"]; data[[4,2]]=1; data[[4,3]]=1; data[[4,4]]=1;  Unfortunately, I have no idea how to proceed further. I found myself very ignorant. I searched through the Mathematica.stack but could not follow the answers. I have never done this. Even if a full answer is not given but a hint, I will try to reconstruct the answer or MWE—apologies for the ignorance. • It is not clear how {{0,:,1,2,3;} maps to {{0,1,1,1,...,0},...}. What are the ...? What does {1,:,0,164,165;} map to? Jun 1, 2021 at 14:26 • @RohitNamjoshi I edited my question. To be quick, the total lattice sites in the above case are 165, so the matrix would be 165 × 165. So, this {{0,1,1,1,...,0},...} means the site at 0 is not connected to any other lattice sites after the third one, hence the zeroes to denote it. Jun 1, 2021 at 14:50 ## 4 Answers 165 × 165 cannot be correct since indexes start at 0 and the last index is 167 so it has to be 168 x 168. rawData = Import["~/Downloads/file.dat", "Data"]; data = Take[rawData, {4, -2}] (* Ignore first 3 and last line *) (* Function to convert site spec like "{0,:,1,2,3;}" to array *) expandRow[spec_] := Module[ {specList = (spec // DeleteCases[":"]) /. x_?StringQ :> ToExpression@StringTake[x, 1 ;; -2]}, SparseArray[Thread[Rest[specList + 1] -> 1], 168] // Normal]  Then map it over the data expandRow /@ data  • Many thanks! This completely solves my problem. Jun 1, 2021 at 22:46 • You are welcome. Thanks for the accept! Jun 2, 2021 at 1:59 you must adapt the code to different forms of input. Here is code that reads the new file: (*input=StringToStream[str];*) input = OpenRead["FileName"]; line = ReadLine[input]; n = StringCases[line, NumberString][[1]] // ToExpression;; dat = Reap[ Do[ line = Read[input, "String"]; Sow[StringCases[line, NumberString] // ToExpression]; , {i, n}] ][[2, 1]]; Close[input]; dat += 1; n = Max[dat]; mat = ConstantArray[0, {n, n}]; Scan[Function[x, (mat[[x[[1]], #]] = 1) & /@ Rest[x]], dat]; mat  • Many thanks for your time! This also completely solves my problem. I agree with you that the code has to be changed according the file. Jun 2, 2021 at 15:47 I assume you will import your matrix as a string, since : is not allowed as an entry in a Mathematica list. If you do, you can use string operations to single out the commands, transform the string to normal Mathematica expressions and use them as you want. For instance, consider the following simple procedure (careful, I use s = s+1 to shift to indices / lattice sites beginning from 1 instead of 0 as in your example) n = 7; matrix = ConstantArray[0, {n, n}]; strings = "{{0,:,1,3,6;},{2,:,1,0,5,3;},{4,:,2,0;}}"; strings = StringSplit[strings, {",{"}]; Do[ s = StringDelete[s, {"{{", ";}}", ";}", ":"}]; s = StringSplit[s, ","]; s = ToExpression@{s[[1]], s[[3 ;;]]}; s = s + 1; Print@InputForm@s; matrix[[s[[1]], s[[2]]]] = 1; , {s, strings} ] MatrixForm@matrix smatrix = SparseArray@matrix; MatrixForm@Normal@smatrix  Since your matrices will be large, consider using SparseArray (take a look into the documentation, I am sure you will find out how to change the code above for that). • Many thanks for your answer! I think I am exporting it as a file, that may be the reason I am getting errors, let me try with the strings. Jun 1, 2021 at 23:11 For an example, we will read the data from a string. If you want to read from a file, activate the second statement and inactivate the first one: str = " !Hello 1. n=11 =0 g 0:1 2 3; 1:0 164 165; 2:0 3 167; 3:0 2 166; 4:5 6 107; 167:2 37 44;$$";

input = StringToStream[str];


Then we read away the first 3 lines:

Do[Read[input, "String", NullRecords -> True], 3];


Next we read all the data and close the input stream:

line = Read[input, "String"];
dat = Reap[
While[line =!= "$$", Sow[ReadList[StringToStream[line], Number, 10^4, RecordSeparators -> {" ", ":", ";"}]]; line = Read[input, "String"]; ]][[2, 1]]; Close[input];  As the data is zero based, we need to add 1; dat += 1;  Then we create a matrix of zeros with a dimension that corresponds to the largest data value: n = Max[dat]; mat = ConstantArray[0, {n, n}];  As far, the data is contained in strings that we need to interpret and set the corresponding matrix elements to one: Scan[Function[x, (mat[[x[[1]], #]] = 1) & /@ Rest[x]], dat];  The result is contained in the variable mat, that is too large to be displayed in full, so we only show the beginning: For convenience, here is the full code: input = StringToStream[str]; (*input=OpenRead[FileName];*) Do[Read[input, "String", NullRecords -> True], 3] line = Read[input, "String"]; dat = Reap[ While[line =!= "$$",