Usage displays properly only after second call

Question

If I have a usage message containig two-dimensional stuff (subscripts or formulas) and I call the usage, then sometimes I get an improperly formatted output enclosed in "...". Linebreaks are gone, but interestingly some formulas survive. Another ?example outputs fine.

It's hard to give an example, because it happens randomly.
Is this a known issue and and if so, is there a fix?

This is the output, when evaluated the first (bad) and second time (ok):

And this is the original usage:

FlaechenPlot::usage=
"FlaechenPlot[{\!\(\*SubscriptBox[\(f\), \(1\)]\), \!\(\*SubscriptBox[\(f\), \(2\)]\),...}, {x, \!\(\*SubscriptBox[\(x\), \(min\)]\), \!\(\*SubscriptBox[\(x\), \(max\)]\)}, {\!\(\*SubscriptBox[\(f\), \(min\)]\), \!\(\*SubscriptBox[\(f\), \(max\)]\)}] zeichnet die Funktionen \!\(\*SubscriptBox[\(f\), \(i\)]\) im Bereich zwischen \!\(\*SubscriptBox[\(x\), \(min\)]\) und \!\(\*SubscriptBox[\(x\), \(max\)]\) und f\[ADoubleDot]rbt den Bereich zwischen \!\(\*SubscriptBox[\(f\), \(min\)]\) und \!\(\*SubscriptBox[\(f\), \(max\)]\) gem\[ADoubleDot]\[SZ] der gew\[ADoubleDot]hlten Option Filling.
FlaechenPlot[f, {x, \!\(\*SubscriptBox[\(x\), \(min\)]\), \!\(\*SubscriptBox[\(x\), \(max\)]\)}, {\!\(\*SubscriptBox[\(f\), \(min\)]\), \!\(\*SubscriptBox[\(f\), \(max\)]\)}] zeichnet entprechend den Bereich zwischen f und der x\[Dash]Achse.
FlaechenPlot[{\!\(\*SubscriptBox[\(f\), \(1\)]\), \!\(\*SubscriptBox[\(f\), \(2\)]\),...}, {x, \!\(\*SubscriptBox[\(x\), \(min\)]\), \!\(\*SubscriptBox[\(x\), \(max\)]\)}] zeichnet und f\[ADoubleDot]rbt das Gebiet zwischen den Kurven im Bereich von \!\(\*SubscriptBox[\(x\), \(min\)]\) bis \!\(\*SubscriptBox[\(x\), \(max\)]\).
FlaechenPlot[f, {x, \!\(\*SubscriptBox[\(x\), \(min\)]\), \!\(\*SubscriptBox[\(x\), \(max\)]\)}] f\[ADoubleDot]rbt und zeichnet das Gebiet zwischen Kurve und x\[Dash]Achse im Bereich von \!\(\*SubscriptBox[\(x\), \(min\)]\) bis \!\(\*SubscriptBox[\(x\), \(max\)]\).
OPTIONEN sind alle Optionen von Plot. Die Plot\[Dash]Option Filling \[Rule] Axis ist default."

Answer 1

I think I have found a solution to this issue, which has been a problem since at least Mathematica version 6 and continues through at least version 11.0. The problem occurs when a user-defined usage message has complex formatting, for example, subscripts, entered using 2D input.

Suppose we have a function with usage message defined as

which also can be input as

f::usage = "\!\(\*SubscriptBox[\(test\), \(1\)]\)\n\!\(\*SubscriptBox[\(test\), \(2\)]\)"

If we evaluate ?f once, we get

which is incorrectly formatted, but if we evaluate it again in the same input cell, without deleting the output cell, we get the correct version:

To see what's going on, we can run Trace[Information[f]], but that produces an enormous output, most of which has nothing to do with formatting the output string. To look for mentions of the usage string itself, we can use

Trace[Information[f], x_String /; !StringFreeQ[x, "test"], TraceBackward -> True] 

This fingers the function System`Dump`fixmessagestring as a possible culprit. The function definition can be found with

InputForm@Definition[System`Dump`fixmessagestring]

which returns

System`Dump`fixmessagestring[System`Dump`s_] :=
  StringJoin[
    "\"",
    StringReplace[System`Dump`s, {"\n" -> "\\n", "\\" -> "\\\\", "\"" -> "\\\""}], 
    "\""
  ]

This function converts some special characters in the string to explict form, as is done in InputForm. However, there are many kinds of formatting that this function doesn't address, which may be part of the problem. We can redefine this function to do a more thorough job by evaluating

System`Dump`fixmessagestring[System`Dump`s_] := ToString@InputForm@System`Dump`s

and now ?f returns the correct result every time. It's still not clear to me why the original version would cause a problem on the first evaluation, but not on the second, but, regardless, it seems that we have a fix.

A concern might be if this has an effect on built-in usage messages, but in my testing, the built-in messages display identically.

The redefinition could be done inside the user-defined package, making the fix invisible.

Also, with the command On[System`Dump`fixmessagestring], we can have Mathematica send us a trace message whenever System`Dump`fixmessagestring is called. That will let us see if the function is ever called anywhere outside of Information[], where the redefinition might have other consequences. So far it seems that the function is just confined to Information[] and ?.

Since there seems to be a one-line fix to this problem, it would be nice if it were eventually fixed in Mathematica itself.

Answer 2

As I mentioned here, this particular issue is fixed in our next release (11.2). This issue was noticed internally in one of the beta release of V6 (but was never reported to support by external users), but it wasn't entirely properly classified for a long time. The reason for that is that this bug is actually two bugs. The understand why, it is helpful to go through the history of this feature.

In version 5, we added hyperlinks from the usage message to the help browser for system functions. The fixmessagestring was added as part of that code. I'm not entirely sure why the author wrote it the way it did (though I have some guesses), but the key point is that--as @simon pointed out--it is buggy. It assumes that the number of backslashes in is independent of the position of the character. But this is not the case--if the string contains (possibly nested) formatted strings (aka "linear syntax"), the number of backslashes will depend on the nesting.

Which is where the second bug comes in. If the kernel sends non-canonical box expressions to the frontend, that is technically undefined behavior. Still, the frontend tries to be nice and fix up the expression and display it--if it can't, you will get the "pink box" behavior of malformed expressions. In version 5, this cleaning up process automatically dealt with the technically incorrect box expressions (a string is a type of box expression) sent by fixmessagestring. In V6, this process fails the first time, but succeeds the second time. This indeterminancy is obviously a bug, but of relatively low priority (and actually the cause still isn't entirely clear) as it is caused by sending invalid boxes to the frontend of a particular type.

The fix then, is for the kernel to send a valid box expression to the frontend. But then kernel already has a function for converting an arbitrary expression to the box language: ToBoxes (or MakeBoxes, depending on context). So the fix was basically to throw out fixmessagestring and to call ToBoxes directly.

I hope you found this illuminating. Let me again stress the importance of sending in reports to support. The more external users complain about something, the more likely it is to received the attention of developers, which can be a rather precious commodity.