Conversion to TeX results in infinite recursion

Question

Bug introduced in 9.0 and persisting through 11.3 or later

Confirmed, as bug, by WRI: [CASE:3059683]

---

I'm trying to export to $\TeX$ a Message cell.

For example, after evaluating Sin[a,b], Sin::argx message is printed. When I try to export cell containing this message to $\TeX$:

ExportString[
    Cell[
        BoxData @ RowBox[{
            StyleBox[RowBox[{"Sin", "::", "argx"}], "MessageName"], 
            RowBox[{":", " "}], 
            "\<\"\[NoBreak]\\!\\(Sin\\)\[NoBreak] called with \[NoBreak]\\!\\(2\\)\[NoBreak] arguments; 1 argument is expected. \\!\\(\\*ButtonBox[\\\"\[RightSkeleton]\\\", ButtonStyle->\\\"Link\\\", ButtonFrame->None, ButtonData:>\\\"paclet:ref/message/General/argx\\\", ButtonNote -> \\\"Sin::argx\\\"]\\)\"\>"
        }],
        "Message",
        "MSG"
    ],
    "TeX"
]

all I get is bunch of $RecursionLimit::reclim and $IterationLimit::itlim errors.

This happens in versions 9 and 10. In version 8 I get correct result.

Minimal non-working example is usage of System`Convert`TeXFormDump`maketex with string containing some special boxes:

System`Convert`TeXFormDump`maketex["\"\\!\\(\\*ButtonBox[\\\"\[RightSkeleton]\\\"]\\)\""]

It also results in $RecursionLimit::reclim and $IterationLimit::itlim errors and unusable result.

How can a message cell be exported to $\TeX$?

Answer 1

It can be traced that, in minimal example from question, following definition of System`Convert`TeXFormDump`maketex function is used:

System`Convert`TeXFormDump`maketex[str_String?System`Convert`CommonDump`EmbeddedStringWithLinearSyntaxQ] := (
    System`Convert`CommonDump`DebugPrint["------------------------------------"];
    System`Convert`CommonDump`DebugPrint["maketex[str_String?EmbeddedStringWithLinearSyntaxQ]"];
    System`Convert`CommonDump`DebugPrint["str: ", str];
    System`Convert`TeXFormDump`MakeTeX[System`Convert`CommonDump`RemoveLinearSyntax[str, System`Convert`CommonDump`Recursive -> True]]
)

Above definition calls System`Convert`CommonDump`RemoveLinearSyntax function with System`Convert`CommonDump`Recursive option. Problem is that, as we can see in function's options:

Options[System`Convert`CommonDump`RemoveLinearSyntax]
(* {System`Convert`CommonDump`ConvertRecursive -> False} *)

it has ConvertRecursive and not Recursive option. At least in versions 9 and 10. In v8 option name is Recursive.

To fix this bug, we can replace option, in maketex definition, with correct one:

Convert`TeX`ExpressionToTeX; (* preload; do not remove! *)

If[FreeQ[Options[System`Convert`CommonDump`RemoveLinearSyntax], System`Convert`CommonDump`Recursive], 
    DownValues[System`Convert`TeXFormDump`maketex] = 
        DownValues[System`Convert`TeXFormDump`maketex] /. 
            Verbatim[System`Convert`CommonDump`RemoveLinearSyntax][arg_, System`Convert`CommonDump`Recursive -> val_] :> 
                System`Convert`CommonDump`RemoveLinearSyntax[arg, System`Convert`CommonDump`ConvertRecursive -> val]
];

It will fix maketex definition, in versions, in which RemoveLinearSyntax has different option than used in maketex.

After above fix, cell, from question, is correctly converted to $\TeX$, in all three tested Mathematica versions.