We can define our own functions.
From $x',y'$ to $r',\theta'$, we derive:
$$
r' = \left(\sqrt{x^2 +y^2} \right)'
= \frac{(x^2 +y^2)'}{2
\sqrt{x^2 +y^2}}=\frac{xx' +yy'}{r}
$$
and
$$
\theta' = \left(\arctan \frac{y}{x} \right)'
= \frac{(y/x)'}{1+(y/x)^2} = \frac{y' x -x' y}{r^2}.
$$
First, we define
rdot[x1_, x2_] := (x1 (μ x1 - x2 - σ x1 (x1^2 + x2^2)) + x2 (x1 + μ x2 - σ x2 (x1^2 + x2^2)))/r
We now make the substitution and simplify
rdot[r Cos[t], r Sin[t]] // FullSimplify
This yields (matches Mathematica)
$$r' = \mu r-r^3 \sigma$$
We now do the same for the other
thetadot[x1_,x2_]:=(x1 (x1+μ x2-σ x2 (x1^2+x2^2)) - x2(μ x1-x2-σ x1 (x1^2+x2^2)))/r^2
We now make the substitution and simplify
thetadot[r Cos[t], r Sin[t]] // FullSimplify
This yields (does not match Mathematica, but see accepted answer)
$$\theta'= 1$$
I have asked this question before on this site in two different ways and have never gotten an answer that resolves the matter, but that could just be my denseness as the accepted answer now shows!
Update
I have received a response from Wolfram Support and wanted to post it as others may find it as helpful as I did.
Thank you for contacting Wolfram Technical Support.
I want to highlight a couple more pieces of information that you might find useful.
There is a more comprehensive tutorial on how Mathematica handles coordinate transformations, and particularly how it handles basis transformations for vectors, available at
https://reference.wolfram.com/language/tutorial/ChangingCoordinateSystems.html
Under the section "Relating Orthonormal Bases", the tutorial highlights that the transformation of vectors is given by an orthonormal rotation matrix. In particular, this guarantees that a vector will have the same norm in any coordinate system. So, the vector {0,1,0} in the {r, th, phi} coordinate system must have a norm 1 in the {x,y,z} coordinate system.
In the question you posted on StackExchange, the norm of the original vector is
r Sqrt[1 + (\[Mu] - r^2 \[Sigma])^2]
(after the change of variables from {x1,x2} to {r,theta} has been made).
This highlights that the proposed solution
{r ? - r^3 ?, 1}
cannot be correct, as it has a different norm. On the other hand,
{r ? - r^3 ?, r}
has the same norm.
Please let me know if you have any further questions.
Sincerely,
Wolfram Technology Group
http://www.wolfram.com/support/