Why doesn't the matrix rank decrease in this case?

Question

Consider the square matrix $M(k)$ of dimension 6

$ M(k) = \frac{1}{\sqrt{2}}\begin{bmatrix} 0 & 0 & 1 & i & 0 & 0\\ 0 & 0 & 0 & 0 & ie^{-ik} & e^{-ik} \\ 0 & 0 & 0 & 0 & e^{i\frac{2\pi}{3}} & ie^{i\frac{2\pi}{3}} \\ ie^{i\frac{2\pi}{3}} & e^{i\frac{2\pi}{3}} & 0 & 0 & 0 & 0\\ e^{ik}e^{-i\frac{2\pi}{3}}& ie^{ik}e^{-i\frac{2\pi}{3}} & 0 & 0 & 0 & 0 \\ 0 & 0 & ie^{-i\frac{2\pi}{3}} & e^{-i\frac{2\pi}{3}} & 0 & 0 \\ \end{bmatrix}$

The code for this matrix is at the end of the question. One of the eigenvalues $\lambda(k)$ is of the form

$\lambda = \frac{ [\sqrt{2}cos(k)-\sqrt{-15+cos(2k)} ]^{1/3} }{ 2^{2/3} }$

The characteristic matrix $C$ of $M$ is defined as

$C(k) \equiv M(k) - I\lambda(k)$

The rank of $C$ should be less than its dimension because $Det[C]=0$, i.e. in this case the rank of $C$ should be less than 6. But, if you compute MatrixRank[$C$] in Mathematica, it returns 6. On the other hand, if you compute Det[$C$] and then fully simplify the expression, it gives you zero.

So there is clearly a problem here. Mathematica knows that the determinant of the matrix $C$ is zero, but it can't see that its rank is lower than its dimension.

As a consequence, Mathematica returns nothing when you ask it for the NullSpace of $C$. This is what I want, the NullSpace of $C$. Any advice on how to solve this?

By the way, if you compute MatrixRank[C] for a particular value of k, then Mathematica returns 5 (which is the correct answer). But for unspecified k, it gives 6 (which is the incorrect answer).

M = {{0,0,1/Sqrt[2],I/Sqrt[2],0,0},
 {0,0,0,0,(I E^(-I k))/Sqrt[2],E^(-I k)/Sqrt[2]},
 {0,0,0,0,E^((2 I π)/3)/Sqrt[2],(I E^((2 I π)/3))/Sqrt[2]},
 {(I E^((2 I π)/3))/Sqrt[2],E^((2 I π)/3)/Sqrt[2],0,0,0,0},
 {E^(I k-(2 I π)/3)/Sqrt[2],(I E^(I k-(2 I π)/3))/Sqrt[2],0,0,0,0},
 {0,0,(I E^(-((2 I π)/3)))/Sqrt[2],E^(-((2 I π)/3))/Sqrt[2],0,0}}

\[Lambda] = (Sqrt[2]Cos[k]-Sqrt[-15+ Cos[2k]])^(1/3)/(2^(2/3))

Answer 1

You can pass in a stronger ZeroTest:

c = M - λ IdentityMatrix[6];

MatrixRank[c, ZeroTest -> PossibleZeroQ @* Simplify]

5

In fact there's a similar example here.