30
$\begingroup$

If I have a usage message containig two-dimensional stuff (subscripts or formulas) and I call the usage, then sometimes I get an improperly formatted output enclosed in "...". Linebreaks are gone, but interestingly some formulas survive. Another ?example outputs fine.

It's hard to give an example, because it happens randomly.
Is this a known issue and and if so, is there a fix?

This is the output, when evaluated the first (bad) and second time (ok):

images of output

And this is the original usage:

FlaechenPlot::usage=
"FlaechenPlot[{\!\(\*SubscriptBox[\(f\), \(1\)]\), \!\(\*SubscriptBox[\(f\), \(2\)]\),...}, {x, \!\(\*SubscriptBox[\(x\), \(min\)]\), \!\(\*SubscriptBox[\(x\), \(max\)]\)}, {\!\(\*SubscriptBox[\(f\), \(min\)]\), \!\(\*SubscriptBox[\(f\), \(max\)]\)}] zeichnet die Funktionen \!\(\*SubscriptBox[\(f\), \(i\)]\) im Bereich zwischen \!\(\*SubscriptBox[\(x\), \(min\)]\) und \!\(\*SubscriptBox[\(x\), \(max\)]\) und f\[ADoubleDot]rbt den Bereich zwischen \!\(\*SubscriptBox[\(f\), \(min\)]\) und \!\(\*SubscriptBox[\(f\), \(max\)]\) gem\[ADoubleDot]\[SZ] der gew\[ADoubleDot]hlten Option Filling.
FlaechenPlot[f, {x, \!\(\*SubscriptBox[\(x\), \(min\)]\), \!\(\*SubscriptBox[\(x\), \(max\)]\)}, {\!\(\*SubscriptBox[\(f\), \(min\)]\), \!\(\*SubscriptBox[\(f\), \(max\)]\)}] zeichnet entprechend den Bereich zwischen f und der x\[Dash]Achse.
FlaechenPlot[{\!\(\*SubscriptBox[\(f\), \(1\)]\), \!\(\*SubscriptBox[\(f\), \(2\)]\),...}, {x, \!\(\*SubscriptBox[\(x\), \(min\)]\), \!\(\*SubscriptBox[\(x\), \(max\)]\)}] zeichnet und f\[ADoubleDot]rbt das Gebiet zwischen den Kurven im Bereich von \!\(\*SubscriptBox[\(x\), \(min\)]\) bis \!\(\*SubscriptBox[\(x\), \(max\)]\).
FlaechenPlot[f, {x, \!\(\*SubscriptBox[\(x\), \(min\)]\), \!\(\*SubscriptBox[\(x\), \(max\)]\)}] f\[ADoubleDot]rbt und zeichnet das Gebiet zwischen Kurve und x\[Dash]Achse im Bereich von \!\(\*SubscriptBox[\(x\), \(min\)]\) bis \!\(\*SubscriptBox[\(x\), \(max\)]\).
OPTIONEN sind alle Optionen von Plot. Die Plot\[Dash]Option Filling \[Rule] Axis ist default."
$\endgroup$
  • $\begingroup$ I have seen this problem myself several times. I can confirm that this happens. $\endgroup$ – Szabolcs Apr 5 '12 at 19:27
  • $\begingroup$ I'm actually seeing a similar problem with Options and SyntaxInformation for a function I've defined in a custom package. When first loading a package, all the options and inputs work, but aren't highlighted properly. Loading the package twice fixes things... $\endgroup$ – tkott Apr 6 '12 at 16:34
  • $\begingroup$ Hmm, only happens to me when restarting frontend. If you restart the kernel, things are fine... $\endgroup$ – tkott Apr 6 '12 at 17:12
  • $\begingroup$ Trace@?example is fascinating. So is ctrl+shift+e: the difference is the addition of extra slashes `` for every one present. $\endgroup$ – tkott Apr 6 '12 at 17:27
27
$\begingroup$

I think I have found a solution to this issue, which has been a problem since at least Mathematica version 6 and continues through at least version 11.0. The problem occurs when a user-defined usage message has complex formatting, for example, subscripts, entered using 2D input.

Suppose we have a function with usage message defined as

Mathematica graphics

which also can be input as

f::usage = "\!\(\*SubscriptBox[\(test\), \(1\)]\)\n\!\(\*SubscriptBox[\(test\), \(2\)]\)"

If we evaluate ?f once, we get

Mathematica graphics

which is incorrectly formatted, but if we evaluate it again in the same input cell, without deleting the output cell, we get the correct version:

Mathematica graphics

To see what's going on, we can run Trace[Information[f]], but that produces an enormous output, most of which has nothing to do with formatting the output string. To look for mentions of the usage string itself, we can use

Trace[Information[f], x_String /; !StringFreeQ[x, "test"], TraceBackward -> True] 

Mathematica graphics

This fingers the function System`Dump`fixmessagestring as a possible culprit. The function definition can be found with

InputForm@Definition[System`Dump`fixmessagestring]

which returns

System`Dump`fixmessagestring[System`Dump`s_] :=
  StringJoin[
    "\"",
    StringReplace[System`Dump`s, {"\n" -> "\\n", "\\" -> "\\\\", "\"" -> "\\\""}], 
    "\""
  ]

This function converts some special characters in the string to explict form, as is done in InputForm. However, there are many kinds of formatting that this function doesn't address, which may be part of the problem. We can redefine this function to do a more thorough job by evaluating

System`Dump`fixmessagestring[System`Dump`s_] := ToString@InputForm@System`Dump`s

and now ?f returns the correct result every time. It's still not clear to me why the original version would cause a problem on the first evaluation, but not on the second, but, regardless, it seems that we have a fix.

A concern might be if this has an effect on built-in usage messages, but in my testing, the built-in messages display identically.

The redefinition could be done inside the user-defined package, making the fix invisible.

Also, with the command On[System`Dump`fixmessagestring], we can have Mathematica send us a trace message whenever System`Dump`fixmessagestring is called. That will let us see if the function is ever called anywhere outside of Information[], where the redefinition might have other consequences. So far it seems that the function is just confined to Information[] and ?.

Since there seems to be a one-line fix to this problem, it would be nice if it were eventually fixed in Mathematica itself.

$\endgroup$
  • $\begingroup$ This is truly a very interesting discovery! How did you find the fixmessagestring function? $\endgroup$ – Silvia Apr 11 '15 at 13:07
  • 1
    $\begingroup$ @Silvia I edited my answer to describe how I found it. $\endgroup$ – Simon Rochester Apr 16 '15 at 0:43
  • $\begingroup$ Thanks! It's very helpful to know the process! :) $\endgroup$ – Silvia Apr 16 '15 at 8:03
  • $\begingroup$ +1 for that last line, almost three years on. $\endgroup$ – Emilio Pisanty May 4 '16 at 21:37
6
$\begingroup$

As I mentioned here, this particular issue is fixed in our next release (11.2). This issue was noticed internally in one of the beta release of V6 (but was never reported to support by external users), but it wasn't entirely properly classified for a long time. The reason for that is that this bug is actually two bugs. The understand why, it is helpful to go through the history of this feature.

In version 5, we added hyperlinks from the usage message to the help browser for system functions. The fixmessagestring was added as part of that code. I'm not entirely sure why the author wrote it the way it did (though I have some guesses), but the key point is that--as @simon pointed out--it is buggy. It assumes that the number of backslashes in is independent of the position of the character. But this is not the case--if the string contains (possibly nested) formatted strings (aka "linear syntax"), the number of backslashes will depend on the nesting.

Which is where the second bug comes in. If the kernel sends non-canonical box expressions to the frontend, that is technically undefined behavior. Still, the frontend tries to be nice and fix up the expression and display it--if it can't, you will get the "pink box" behavior of malformed expressions. In version 5, this cleaning up process automatically dealt with the technically incorrect box expressions (a string is a type of box expression) sent by fixmessagestring. In V6, this process fails the first time, but succeeds the second time. This indeterminancy is obviously a bug, but of relatively low priority (and actually the cause still isn't entirely clear) as it is caused by sending invalid boxes to the frontend of a particular type.

The fix then, is for the kernel to send a valid box expression to the frontend. But then kernel already has a function for converting an arbitrary expression to the box language: ToBoxes (or MakeBoxes, depending on context). So the fix was basically to throw out fixmessagestring and to call ToBoxes directly.

I hope you found this illuminating. Let me again stress the importance of sending in reports to support. The more external users complain about something, the more likely it is to received the attention of developers, which can be a rather precious commodity.

$\endgroup$
  • $\begingroup$ Could you provide an example of a string with nested "formatted strings (aka "linear syntax")"? Currently I can't imagine what is it, but it may help testing correctness of my ImportAsASCII function. Also I would be grateful to receive a response to that question from you. $\endgroup$ – Alexey Popkov Jul 7 '17 at 4:02
  • 1
    $\begingroup$ @AlexeyPopkov, a formatted string, which we internally call linear syntax, is a string which display as a 2-d formatted expression. A simple example is the following "\!\(a\^b\)". If you copy and paste this into an Input cell and evaluate it, you'll see an a with a superscripted b. The formatted sections of these strings always started with \!\( and end with \). Inside of these delimieters, portions of the string are interpreted specially. See tutorial/InputSyntax#29846. All of the strings affected by this bug used linear syntax, either for 2d typesetting, or styles like italics. $\endgroup$ – Itai Seggev Jul 7 '17 at 22:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.