4
$\begingroup$

I'm trying to solve the following equation for small values of $\epsilon$, ie, $\epsilon\approx 10^{-12}$.

$$e^{-t \epsilon } \text{csch}(t) \sinh (t-t \epsilon ) = \epsilon$$

Using FindRoot below suffers from numerical precision issues, which don't appear to be addressed by extended precision. IE, using 100 digits of precision works for $\epsilon= 10^{-6}$, but gets stuck for $\epsilon=10^{-9}$

loss = E^(-ε t) Csch[t] Sinh[t - ε t];

prec = 100; setPrec[t_] := SetPrecision[t, prec + 1];
findSteps[e0_] := 
  Block[{ε = setPrec[e0]}, 
   t /. FindRoot[loss == ε, {t, 1/ε}, 
     WorkingPrecision -> prec]];

findSteps[10^-6]

Any tips?

Notebook

Improve this question
$\endgroup$
14
  • $\begingroup$ Isn't it art for art's sake? $\endgroup$
    – user64494
    Commented Jun 19, 2023 at 5:47
  • $\begingroup$ @user64494 the motivation is that it gives an upper bound on the number of gradient descent steps until loss is numerically 0, for an arbitrary quadratic objective, in limited precision determined by $\epsilon$ $\endgroup$
    – Yaroslav Bulatov
    Commented Jun 19, 2023 at 5:54
  • $\begingroup$ Sorry, don't understand you. Please, state your motivation clearly. $\endgroup$
    – user64494
    Commented Jun 19, 2023 at 5:57
  • $\begingroup$ The problem is equivalent to finding $t$ such that almost all of integral mass $\int_0^1 \exp(-2 t x)\mathrm{d}x$, is contained in the interval $[0,\epsilon]$. This integral comes up in analysis of gradient descent $\endgroup$
    – Yaroslav Bulatov
    Commented Jun 19, 2023 at 6:04
  • $\begingroup$ Still don't understand what for. $\endgroup$
    – user64494
    Commented Jun 19, 2023 at 6:11

3 Answers 3

Reset to default
8
$\begingroup$

Asymptotic analysis along the lines of @Roland's answer reveals that the root of $$e^{-t \epsilon } \text{csch}(t) \sinh (t-t \epsilon ) = \epsilon$$ as $\epsilon\rightarrow 0$ is at $t \sim \frac{1}{2\epsilon}\ln(1/\epsilon)$.

This code visually confirms it,

With[{ε=10^-p},
   Manipulate[
      Plot[E^(-ε t) Csch[t] Sinh[t-ε t]-ε,
      {t,-10,320},
      PlotRange->{-.5,2},
      Epilog->{Red,AbsolutePointSize[7],Point[{1/(2ε) Log[1/ε],0}]}],
   {p,1,5}]
]

and for $\epsilon = 10^{-9}$, we can easily evaluate this high quality approximation: $t = 1.036\times10^{10}$.

Improve this answer
$\endgroup$
9
  • 1
    $\begingroup$ Substituting $\epsilon = 10^{-12}$ yields a root of $t=1.38\times 10^{13}$, consistent with @user64494's answer based on NMinimize. $\endgroup$
    – QuantumDot
    Commented Jun 19, 2023 at 6:33
  • $\begingroup$ -1. Sorry, "Asymptotic analysis along the lines of @Roland's answer reveals that the root of e−tϵcsch(t)sinh(t−tϵ)=ϵ as ϵ→0 is at t∼12ϵln(1/ϵ)" are empty words. $\endgroup$
    – user64494
    Commented Jun 19, 2023 at 6:43
  • $\begingroup$ Indeed with loss[ε_, t_] = E^(-ε t) Csch[t] Sinh[t - ε t] we have $\varepsilon^{-1}\text{loss}\left(\varepsilon ,-\frac{\ln\varepsilon}{2\varepsilon}\right)=\frac{\varepsilon^{\frac{1}{\varepsilon}-1}-1}{\varepsilon^{\frac{1}{\varepsilon}}-1}$, which goes to 1 very rapidly as $\varepsilon\to0$. Try this: Plot[{loss[ε, -Log[ε]/(2ε)]/ε, (ε^(1/ε-1)-1)/(ε^(1/ε)-1)}, {ε, 0, 0.2}]. $\endgroup$
    – Roman
    Commented Jun 19, 2023 at 6:55
  • 5
    $\begingroup$ @user64494 I understand very well that you understand nothing of these matters. Cheers! $\endgroup$
    – Roman
    Commented Jun 19, 2023 at 7:06
  • 1
    $\begingroup$ @Roman: \[CurlyEpsilon]^(-1)* loss[\[CurlyEpsilon], -Log[\[CurlyEpsilon]]/(2 \[CurlyEpsilon])] results in -((Csch[Log[\[CurlyEpsilon]]/(2 \[CurlyEpsilon])] Sinh[ Log[\[CurlyEpsilon]]/2 - Log[\[CurlyEpsilon]]/( 2 \[CurlyEpsilon])])/Sqrt[\[CurlyEpsilon]]). $\endgroup$
    – user64494
    Commented Jun 19, 2023 at 7:36
2
$\begingroup$

Approximate up to a certain order in $\varepsilon$

 f[t_] = ε /. Solve[Series[E^(-ε t) Csch[t] Sinh[t - ε t] == ε, {ε, 0, 3}] // Normal, ε]

and look for the real solution

 Plot[Evaluate[ReIm[f[t]]][[1]], {t, -4, 4}]

enter image description here

FindRoot[(E^(-ε t) Csch[t] Sinh[t - ε t] - ε ) /. {t -> 2}, {ε, 0.3}]
(* {ε->0.294713} *)
Improve this answer
$\endgroup$
4
  • $\begingroup$ So the issue is precision when $t\approx 10^{12}$ or $\epsilon\approx 10^{-12}$. I tried series expansion around $t=\infty$ but it returns unevaluated $\endgroup$
    – Yaroslav Bulatov
    Commented Jun 19, 2023 at 5:36
  • $\begingroup$ All that approximation is built on the sand. $\endgroup$
    – user64494
    Commented Jun 19, 2023 at 5:58
  • $\begingroup$ We usually call it an approximation of 0 by very small values of 1. $\endgroup$
    – Roland F
    Commented Jun 19, 2023 at 6:05
  • $\begingroup$ In particular, Limit[Series[ E^(-\[CurlyEpsilon] t) Csch[t] Sinh[ t - \[CurlyEpsilon] t] - \[CurlyEpsilon], {\[CurlyEpsilon], 0, 3}] // Normal, t -> Infinity, Assumptions -> \[CurlyEpsilon] > 0] produces -Infinity, whereas Limit[E^(-\[CurlyEpsilon] t) Csch[t] Sinh[ t - \[CurlyEpsilon] t] - \[CurlyEpsilon], t -> Infinity, Assumptions -> \[CurlyEpsilon] > 0] results in -\[CurlyEpsilon]. $\endgroup$
    – user64494
    Commented Jun 19, 2023 at 6:10
-2
$\begingroup$

Making use of NMinimize instead of FindRoot, one succeeds.

loss1 = E^(-\[CurlyEpsilon] t) Csch[t] Sinh[t - \[CurlyEpsilon] t] - \[CurlyEpsilon];
\[CurlyEpsilon] = 10^-12; NMinimize[{loss1^2, t >= 10^3}, t, WorkingPrecision -> 30]

{1.90868331977198098151173852671*10^-52, {t -> 1.38155105579642671963526697470*10^13}}

NMinimize[{loss1^2, t >= 10^3}, t, WorkingPrecision -> 100]

{1.9086833197722663574515674286889244248591654406882070452413208387700\ 16934234187487647725103961189817*10^-122, {t -> 1.38155105579642741041079487281061852456066089317725687786471775836\ 5547144183076004688106490290838025*10^13}}

Improve this answer
$\endgroup$
3
  • $\begingroup$ What is wrong in my answer? $\endgroup$
    – user64494
    Commented Jun 19, 2023 at 6:50
  • 8
    $\begingroup$ Probably nothing. The comments are another matter entirely. They are rude and mostly off-target. Again, this is not a mathematics forum, it is a forum for computing with a particular software product. Big difference. $\endgroup$
    – Daniel Lichtblau
    Commented Jun 19, 2023 at 22:27
  • 4
    $\begingroup$ I was reluctant to explain my –1 vote because you have a history of not accepting criticism. It's a terrible idea to convert a zero crossing into a minimum and then using a minimization algorithm to find it. Root-finding with Newton's method or other methods is much more efficient than minimum-finding: the error signal of a root finder is first-order, whereas the error signal of a minimizer is second-order. And if the minimum-finder differentiates the argument to convert it back to a root-finding problem, then it simply reverses the conversion $\endgroup$
    – Roman
    Commented Jun 20, 2023 at 9:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.