Sorting Data by Differences between Columns

Question

In Band Structure calculations one gets rather extensive data of Energy Values (Eigenvalues) and the associated Eigenvectors out of the solution of an Eigenequation of e.g. the form H.$\psi$=E.$\psi$. Solving this equation yield usually Eigenvalues/Eigenvector pairs sorted by decreasing Eigenvalues.

An example of such an dataset is shown here

data={{{-9.16206, -7.58803, -4.31229, -2.85908, 4.0047, 4.33906, 6.60042, 
7.32482}, {{0.659095 - 0.0205512 I, -0.0074541 + 0.000232426 I, 
0.0000771943 + 0.0608106 I, -0.00371154 - 0.0606973 I, 
0.480582 + 0.451518 I, 
0.0054352 + 0.00510649 I, -0.040295 + 0.0455439 I, 
0.042945 - 0.0430542 I}, {-0.569797 + 0.159891 I, -0.132206 + 
 0.0370984 I, -0.146127 + 0.0289249 I, -0.139842 + 0.0513211 I, 
0.515967 + 0.289847 I, -0.119716 - 0.0672513 I, -0.135173 - 
 0.062594 I, -0.12378 - 0.0828745 I}, {0.022029 + 
 0.102154 I, -0.129324 - 0.599706 I, 
0.323283 - 0.408073 I, -0.462729 - 0.238574 I, 
0.056657 - 0.0878108 I, 
0.332611 - 0.515502 I, -0.158502 - 0.495896 I, 
0.517147 - 0.0599557 I}, {0.00135768 - 0.0109025 I, 
0.0677891 - 0.544363 I, -0.285998 + 0.275605 I, 
0.209669 + 0.33733 I, 
0.00866928 - 0.00674923 I, -0.432857 + 0.336988 I, 
0.0902696 - 0.386787 I, 
0.397113 + 0.00734894 I}, {0.478682 - 0.125004 I, 
0.100588 - 0.0262677 I, -0.332849 + 0.337306 I, -0.455272 - 
 0.131494 I, -0.426871 - 0.250088 I, 
0.0897003 + 0.0525522 I, -0.228946 - 0.414906 I, -0.473871 + 
 0.0031511 I}, {-0.0926624 - 0.0934746 I, 
0.313489 + 0.316237 I, -0.282601 + 0.246448 I, 
0.248905 - 0.28044 I, 0.000574334 - 0.131619 I, 
0.00194305 - 0.445284 I, -0.374303 + 0.0222986 I, 
0.374094 + 0.0255643 I}, {-0.448984 + 0.190802 I, -0.354137 + 
 0.150495 I, -0.282038 + 0.29955 I, -0.411402 - 
 0.00486289 I, -0.452397 - 0.182562 I, 0.356829 + 0.143996 I, 
0.287467 + 0.294344 I, 
0.411245 - 0.0123828 I}, {0.0966639 + 0.257887 I, -0.242141 - 
 0.646001 I, -0.210795 + 0.17766 I, 0.275637 - 0.0046696 I, 
0.114002 - 0.250705 I, 0.285572 - 0.628011 I, 
0.198206 + 0.191603 I, -0.274679 - 
 0.0234301 I}}}, {{-9.29415, -7.50801, -4.24412, -2.68539, 
3.85009, 4.18365, 6.82186, 
7.26021}, {{-0.668047 + 0.0375069 I, 
0.00216553 - 0.000121582 I, -0.00859939 - 
 0.0594807 I, -0.00188735 + 0.0600695 I, -0.498902 - 
 0.445859 I, -0.00161723 - 0.00144529 I, 
0.0438101 - 0.041141 I, -0.0359785 + 0.0481399 I}, {-0.548008 + 
 0.196971 I, -0.158467 + 0.0569581 I, -0.167735 + 
 0.0310185 I, -0.149101 + 0.0828622 I, 
0.52678 + 0.24822 I, -0.152329 - 0.0717778 I, -0.164023 - 
 0.0468378 I, -0.14054 - 0.0966731 I}, {0.0330804 + 
 0.125763 I, -0.153878 - 0.585007 I, 
0.315782 - 0.422264 I, -0.48268 - 0.212239 I, 
0.0655366 - 0.112319 I, 
0.304854 - 0.522471 I, -0.191231 - 0.491382 I, 
0.521878 - 0.0752944 I}, {0.00132524 - 0.009075 I, 
0.0790595 - 0.541385 I, -0.279998 + 0.270281 I, 
0.191015 + 0.339064 I, 
0.00735408 - 0.00547991 I, -0.438721 + 0.326914 I, 
0.104687 - 0.374822 I, 
0.389106 + 0.00687122 I}, {0.478287 - 0.133411 I, 
0.0843676 - 0.0235332 I, -0.342299 + 0.339145 I, -0.46842 - 
 0.113006 I, -0.432536 - 0.243864 I, 
0.0762974 + 0.0430164 I, -0.251315 - 0.41113 I, -0.481853 - 
 0.00223001 I}, {-0.0639162 - 0.0690425 I, 
0.296278 + 0.320041 I, -0.28542 + 0.251055 I, 
0.272286 - 0.265243 I, 0.00362487 - 0.094016 I, 
0.0168028 - 0.435803 I, -0.38009 - 0.00498043 I, 
0.379345 + 0.0243003 I}, {-0.470095 + 0.186804 I, -0.347478 + 
 0.138079 I, -0.29015 + 0.303008 I, -0.418988 - 
 0.021215 I, -0.464498 - 0.200317 I, 0.343341 + 0.148068 I, 
0.281268 + 0.31127 I, 
0.419426 - 0.00909209 I}, {0.101492 + 0.272772 I, -0.241163 - 
 0.648157 I, -0.211764 + 0.174939 I, 0.274611 - 0.00602878 I, 
0.121114 - 0.264644 I, 0.287789 - 0.628845 I, 
0.198442 + 0.189916 I, -0.27344 - 
 0.0260391 I}}}, {{-9.42409, -7.44177, -4.12004, -2.51234, 
3.70499, 4.06284, 7.05022, 
7.10828}, {{-0.677397 + 0.0585892 I, -0.00400919 + 
 0.000346761 I, -0.0171565 - 0.0571935 I, -0.00708171 + 
 0.0592899 I, -0.520421 - 0.437563 I, 0.00308012 + 0.00258973 I, 
0.0469318 - 0.0369168 I, -0.0283104 + 0.0525734 I}, {-0.523318 + 
 0.229985 I, -0.179851 + 0.0790397 I, -0.188072 + 
 0.0368023 I, -0.154295 + 0.113659 I, 
0.532666 + 0.207418 I, -0.183063 - 0.0712842 I, -0.189472 - 
 0.0287341 I, -0.15901 - 0.106964 I}, {-0.0457089 - 0.147101 I, 
0.175658 + 0.565305 I, -0.307154 + 0.434651 I, 
0.499398 + 0.184031 I, -0.071695 + 0.136337 I, -0.275522 + 
 0.52394 I, 
0.222998 + 0.483257 I, -0.524536 + 0.0901544 I}, {0.00117658 - 
 0.0069311 I, 0.0916398 - 0.539838 I, -0.273934 + 0.26512 I, 
0.1711 + 0.340666 I, 
0.005733 - 0.00406906 I, -0.446522 + 0.316924 I, 
0.119902 - 0.361873 I, 
0.381169 + 0.00623288 I}, {-0.480425 + 0.140868 I, -0.06232 + 
 0.0182731 I, 0.354126 - 0.341156 I, 0.48228 + 0.0959093 I, 
0.43932 + 0.240103 I, -0.056988 - 0.0311459 I, 
0.273205 + 0.408841 I, 
0.491638 + 0.00917125 I}, {-0.0413165 - 0.0467935 I, 
0.279035 + 0.316025 I, -0.29069 + 0.258302 I, 
0.292314 - 0.256463 I, 0.00387288 - 0.0623032 I, 
0.0261559 - 0.420771 I, -0.388044 - 0.0253503 I, 
0.388196 + 0.0229022 I}, {0.487796 - 0.187144 I, 
0.344936 - 0.132335 I, 0.296998 - 0.30563 I, 
0.425208 + 0.028554 I, 
0.477254 + 0.212593 I, -0.337482 - 0.150331 I, -0.280477 - 
 0.320858 I, -0.426122 + 0.00610367 I}, {0.101184 + 
 0.282508 I, -0.233844 - 0.652901 I, -0.211332 + 0.170614 I, 
0.271597 - 0.00235316 I, 0.128216 - 0.271311 I, 
0.296318 - 0.627024 I, 
0.193712 + 0.190384 I, -0.270076 - 
 0.0287925 I}}}, {{-9.55153, -7.38605, -3.94069, -2.34127, 
3.57007, 3.9788, 6.87615, 
7.28141}, {{0.489182 - 0.489182 I, 0.00767275 - 0.00767275 I, 
0.0533557 + 0.0265756 I, -0.0265756 - 0.0533557 I, 
0.691807, -0.0108509, -0.0189364 + 0.05652 I, -0.0189364 - 
 0.05652 I}, {-0.396197 + 0.396197 I, -0.155242 + 
 0.155242 I, -0.181657 + 0.106054 I, -0.106054 + 0.181657 I, 
0.560307, -0.219546, -0.203442 + 0.0534591 I, -0.203442 - 
 0.0534591 I}, {-0.0747996 + 0.157997 I, 0.246121 - 0.519874 I, 
0.52566 - 0.103612 I, -0.253027 - 0.472262 I, 
0.164612 - 0.0588292 I, 0.54164 - 0.193572 I, 
0.155023 - 0.512856 I, 
0.444962 + 0.298433 I}, {-0.00338552 + 0.00338552 I, -0.388792 + 
 0.388792 I, 0.367164 - 0.0666031 I, 
0.0666031 - 0.367164 I, -0.00478784, 
0.549835, -0.30672 + 0.212529 I, -0.30672 - 
 0.212529 I}, {-0.485554 + 0.147667 I, -0.0341219 + 0.0103772 I, 
0.36826 - 0.343348 I, 0.497065 + 0.0801855 I, 
0.447755 + 0.238922 I, -0.0314656 - 0.0167901 I, 
0.294778 + 0.408178 I, 
0.503183 + 0.0176153 I}, {-0.00276422 + 0.0352472 I, 
0.0314966 - 0.40162 I, 
0.399837 + 0.0213424 I, -0.398276 - 0.0412485 I, -0.0268782 + 
 0.022969 I, -0.30626 + 0.261717 I, 
0.252456 + 0.31079 I, -0.267636 - 0.297819 I}, {-0.302329, 
0.69608, -0.0903668 - 0.250349 I, -0.0903668 + 0.250349 I, 
0.213779 + 0.213779 I, 
0.492203 + 0.492203 I, -0.240923 + 0.113125 I, 
0.113125 - 0.240923 I}, {0.491619 + 0.220697 I, 
0.337305 + 0.151422 I, 0.431899 - 0.00337925 I, 
0.284492 + 0.32498 I, 
0.191571 + 0.503683 I, -0.131439 - 0.345582 I, 
0.0286294 - 0.430962 I, -0.307788 - 
 0.303009 I}}}, {{-9.67576, -7.33705, -3.70994, -2.17392, 3.44493,
3.9318, 6.57736, 
7.51153}, {{0.704577, 0.018062, 0.0255535 + 0.053998 I, 
0.0255535 - 0.053998 I, 
0.498211 + 0.498211 I, -0.0127718 - 0.0127718 I, -0.0562514 + 
 0.0201133 I, 0.0201133 - 0.0562514 I}, {0.549046, 0.236761, 
0.217128 + 0.0610546 I, 
0.217128 - 0.0610546 I, -0.388234 - 0.388234 I, 
0.167415 + 0.167415 I, 0.196705 + 0.110361 I, 
0.110361 + 0.196705 I}, {0.0713886 + 0.17741 I, -0.207166 - 
 0.514833 I, 0.290505 - 0.453105 I, -0.523374 - 0.125605 I, 
0.0749682 - 0.175927 I, 
0.217554 - 0.510531 I, -0.281265 - 0.458897 I, 
0.525811 - 0.114976 I}, {0.0026574, 
0.553823, -0.306294 - 0.198181 I, -0.306294 + 0.198181 I, 
0.00187907 + 0.00187907 I, -0.391612 - 0.391612 I, 
0.356718 + 0.0764478 I, 
0.0764478 + 0.356718 I}, {-0.458365 - 0.240482 I, 
0.000917614 + 0.000481428 I, 0.51634 + 0.027467 I, 
0.31604 + 0.409243 I, 0.154066 + 0.494159 I, 
0.00030843 + 0.000989272 I, -0.0659045 + 0.512853 I, 
0.345686 + 0.38453 I}, {0.00944186 + 0.0078976 I, -0.291725 - 
 0.244012 I, 0.252309 - 0.328425 I, -0.278668 + 0.306377 I, 
0.00109196 + 0.0122609 I, 0.0337381 + 0.378823 I, 
0.413689 - 0.0195932 I, -0.410641 + 0.0538219 I}, {-0.297446, 
0.699372, -0.0937752 - 0.240596 I, -0.0937752 + 0.240596 I, 
0.210326 + 0.210326 I, 
0.494531 + 0.494531 I, -0.236436 + 0.103818 I, 
0.103818 - 0.236436 I}, {0.508109 - 0.225031 I, 
0.341738 - 0.151349 I, 0.292996 - 0.324207 I, 
0.436986 + 0.000913268 I, 
0.518408 + 0.200166 I, -0.348665 - 0.134626 I, -0.30835 - 
 0.309641 I, -0.436429 + 
 0.0220696 I}}}, {{-9.79568, -7.29167, -3.43432, -2.01255, 
3.32762, 3.92047, 6.23075, 
7.73642}, {{0.50769 - 0.50769 I, 0.0179752 - 0.0179752 I, 
0.0585483 + 0.0133341 I, -0.0133341 - 0.0585483 I, 
0.717982, -0.0254208, -0.0319713 + 0.0508285 I, -0.0319713 - 
 0.0508285 I}, {0.53838, 0.248219, 0.227913 + 0.0644604 I, 
0.227913 - 0.0644604 I, -0.380692 - 0.380692 I, 
0.175518 + 0.175518 I, 0.206739 + 0.115579 I, 
0.115579 + 0.206739 I}, {0.0825849 + 0.185118 I, -0.216398 - 
 0.485065 I, 0.284012 - 0.458976 I, -0.531269 - 0.0952617 I, 
0.0725018 - 0.189294 I, 
0.189977 - 0.496009 I, -0.308304 - 0.443024 I, 
0.525372 - 0.123718 I}, {-0.000587409 + 
 0.000587409 I, -0.395476 + 0.395476 I, 0.345235 - 0.0873177 I, 
0.0873177 - 0.345235 I, -0.000830722, 
0.559288, -0.305861 + 0.182375 I, -0.305861 - 
 0.182375 I}, {-0.506692 + 0.160323 I, 0.0442719 - 0.0140081 I, 
0.402599 - 0.34802 I, 0.529514 + 0.0530883 I, 
0.471651 + 0.24492 I, 0.0412102 + 0.0213997 I, 
0.336884 + 0.411962 I, 
0.530768 + 0.0385935 I}, {0.000647736 - 0.00682218 I, 
0.0334587 - 0.352399 I, 
0.429224 + 0.0176433 I, -0.424875 - 0.0634496 I, 
0.00528203 - 0.00436599 I, -0.272842 + 0.225525 I, 
0.255566 + 0.345297 I, -0.291031 - 0.315983 I}, {-0.284855, 
0.703035, -0.100754 - 0.226709 I, -0.100754 + 0.226709 I, 
0.201423 + 0.201423 I, 
0.497121 + 0.497121 I, -0.231552 + 0.0890637 I, 
0.0890637 - 0.231552 I}, {-0.213247 + 0.532038 I, -0.141676 + 
 0.353472 I, 
0.00887743 + 0.441362 I, -0.311255 + 0.313049 I, -0.526996 + 
 0.225419 I, 0.350123 - 0.149763 I, 0.44145 - 0.00126895 I, 
0.305813 - 0.318367 I}}}}

The data format is a list of Eigenvalues and associated Eigenvectors for (in this example 6) different reciprocal lattice vectors of the band structure. For each of the 6 datapoints there is first a list of (in this case 8) Eigenvalues and second a list of 8 Eigenvectors. One can plot this data easily by applying

ListLinePlot[Transpose[data[[;; , 1]]]]

yielding

or showing the data points with

ListLinePlot[Transpose[data[[;; , 1]]]]

One can clearly see that the two top bands cross over which is not recognizable by the ListPlot routine since it's acting on the sorted Eigenvalues. If looking at the Eigenvectors in reality the two top bands are running from bottom left to top right and vice versa respectively and do not have the discontinuity in slope at their crossing. The characteristics of each Eigenvalue in the band can be determined by the associated (normalized) Eigenvector which has a characteristic combination of components. However these Eigenvectors also change over the running reciprocal vectors and thus such crossings are difficult to detect by just looking at the dedicated datapoints. Now the issue: I would need to sort the Energy Values in an order reflected by the Eigenvectors. E.g. by starting with the first point (the first reciprocal vector) in the dataset, take the canonical order of the Eigenvalues and the associated Eigenvectors as a reference and then use the differences of the Eigenvectors for the next point to determine the right order or Eigenvalues to prevent crossovers like shown in the example data. Thus for the above example the order of the Eigenvalues will stay canonical until the cross-over happens, wheras afterwards the two top Eigenvalues will get flipped. These cross-overs can be best seen by looking at the Norms of the complex components of the last two Eigenvectors of points #3,#4 and #5 of above data

Abs[data[[2, 2, -2 ;;]]]
Abs[data[[3, 2, -2 ;;]]]
Abs[data[[4, 2, -2 ;;]]]

yielding

{{0.505851, 0.373907, 0.419525, 0.419525, 0.505851, 0.373907, 0.419525, 0.419525}, 
 {0.291042, 0.691569, 0.274677, 0.274677, 0.291042, 0.691569, 0.274677, 0.274677}}

{{0.522463, 0.36945, 0.426166, 0.426166, 0.522463, 0.36945, 0.426166, 0.426166}, 
 {0.300081, 0.693515, 0.271607, 0.271607, 0.300081, 0.693515, 0.271607, 0.271607}}

{{0.302329, 0.69608, 0.26616, 0.26616, 0.302329, 0.69608, 0.26616, 0.26616},
 {0.538884, 0.369734, 0.431912, 0.431912, 0.538884, 0.369734, 0.431912, 0.431912}}

The last vector-set indicates that the associated Eigenvalues need to get flipped to have the "natural" sorting.

Is there a simple built in way in Mathematica how to establish such an sorting?

Answer 1

I've apparently figured out how to find which band is which by looking at the eigenvectors:

data // Map[Transpose] // Flatten[#, 1] & // (* structuring the data *)
              Join[{#}, Abs@#2] & @@@ # & // (* construct a list of *)
                                          (* eigenvalues catenated with their *)
                                          (* eigenvectors Abs of each component *)
   FindClusters[#, 8, DistanceFunction -> (Norm@*Rest@*Subtract)] & // 
                                    (* I shall find clusters by the 
                                       closeness of eigenvectors *)
   #[[;; , ;; , ;; 3]] & // ListPointPlot3D (* display *)

Good, we can see that the eight bands are now separable. I need a FindClustersBy function... but that doesn't exist. I also lost the order of the eigenvalues in the process - now I don't know which one is the first, and which is the sixth.

So I'll roll my own FindClustersBy.

idxs = Table[{i, j}, {i, 6}, {j, 8}] // Flatten[#, 1] &;
FindClusters[idxs, 8, 
 DistanceFunction -> (Norm[
     Abs[data[[First@#, 2, Last@#]]] - 
      Abs[data[[First@#2, 2, Last@#2]]]] &)]

Pay close attention to the last two rows of the table. Now I plot this:

Map[data[[First@#,1,Last@#]]&, %, {2}];
ListPlot@%

Now the two top bands correctly cross. If I knew more about the structure of the dataset, I could maybe explain, why I'm comparing the datapoints based on the absolute values of their eigenvector components, rather than the complex-valued vectors. There's probably an arbitrary phase there.

Answer 2

This is how you can do what you are asking for.

np = 6 (*no of k points*)
ne = 8 (*no of bands*)
norm = Table[ Map[Norm[#] &, (data[[i, 2]] - data[[i - 1, 2]])], {i, 2, np}];
norm = Join[{Range[ne]}, norm];
 (*norm contains Norm of difference between the eigen vectors.
   you can use some other quantity as well  *)
data1 = Map[Transpose[Join[{data[[#, 1]]}, {norm[[#]]}]] &, Range[np]];
data2 = Transpose@Table[Sort[data1[[i]], #1[[2]] < #2[[2]] &][[All, 1]], {i, np}]
(*Eigenvaluse sorted according to `norm` values*)

ListLinePlot[data2]