# How can I get the unchanged Box form of an arbitrary expression?

One may observe that MakeBoxes does not give the actual Box form of various expressions:

MakeBoxes[{1*^4, 000123, a*b c}]

RowBox[{"{", RowBox[{"10000", ",", "123", ",", RowBox[{"a", " ", "b", " ", "c"}]}], "}"}]


The 1*^4 was expanded, the leading zeros of 000123 were stripped, and the distinction between * and (space) was lost.

Also, MakeBoxes doesn't like syntactically invalid or incomplete strings:

MakeBoxes[0001+1,*^6*a b]


Syntax::sntxi: Incomplete expression; more input is needed.

How can one get the actual Box form visible with Show Expression (Ctrl+Shift+E) without copy & paste?

-
In my answer here, in the section "Text file procedure" this problem is solved by extracting the boxes from the .nb file. – Jacob Akkerboom May 27 '14 at 13:25

My other answer is a nice solution for interactively looking at boxes, but in the comments, Mr.Wizard seems to be indicating that he's more interested in programmatic usage, and that he's definitely interested in seeing the box form after the FE has stripped non-semantic boxes to send to the kernel. So here's a totally different method for doing this which achieves those goals.

MathLinkCallFrontEnd[
FrontEndUndocumentedTestFEParserPacket["a*b c+d", True]]


The first argument to UndocumentedTestFEParserPacket must be a string, so this solution precludes 2D input unless you formulate the 2D input using linear syntax. The second argument indicates whether the result should strip non-semantic boxes in the same way that the FE does at Shift+Enter time. True indicates that it should strip (note the return value does not include the space between b and c). Replacing it with False would leave the non-semantic boxes exactly as if they were being written to a notebook file.

If you're wondering what a non-semantic box is, this includes non-semantic spaces and several different box types if they have StripOnInput set to true. The list, and default values of StripOnInput options can be found in the Option Inspector (just search for StripOnInput). StyleBox also takes the StripOnInput option. By default, StyleBox is stripped in math but not in 2D or 3D graphics. Here's an example of StyleBox stripping.

MathLinkCallFrontEnd[
FrontEndUndocumentedTestFEParserPacket[
"\!$$\*StyleBox[\"x\",\"style\"]$$", True]]


returns just the x while

MathLinkCallFrontEnd[
FrontEndUndocumentedTestFEParserPacket[
"\!$$\*StyleBox[\"x\",\"style\",StripOnInput->False]$$", True]]


returns the full StyleBox.

-
Thank you, thank you, thank you!!! – halirutan Oct 20 '12 at 22:39
Welcome back around – Rojo Oct 21 '12 at 0:10
This looks like the function I thought must exist. The name is amazing. Is there a reason you don't use FrontEndExecute? – Mr.Wizard Oct 21 '12 at 6:00
@Mr.Wizard, FrontEndExecute is implemented in terms of MathLinkCallFrontEnd, so the kernel definitions are functionally equivalent. I have reasons for preferring MathLinkCallFrontEnd, but they're quirky, and most people probably wouldn't find them relevant. I can't take credit for the name...I think Jason Harris came up with it. – John Fultz Oct 21 '12 at 6:20

Here's some code which produces an InputField and the box form of anything you type into the InputField as you type it:

DynamicModule[{boxes = ""},
Column[{InputField[Dynamic[boxes], Boxes, ContinuousAction -> True],
Dynamic[boxes, BaseStyle -> {ShowStringCharacters -> True}]}]]


The critical idea here is using a Boxes style InputField to get the box form, which you can then work with programmatically.

I might have come up with this idea on my own, but I should give credit to Jason Harris who gave a typesetting workshop today and had an example very close to this, so this was on my mind.

-
This is very helpful, thank you. I notice that when I paste {1*^4, 000123, a*b c} it is displayed as {1.\[Times]10^4, 000123, a*b c} -- is there a way to keep 1*^4 as 1*^4 in the InputField itself? Also, I notice that this preserves the spaces e.g. "b", " ", "c" while $PreRead does not; are you able to comment on that difference? – Mr.Wizard Oct 20 '12 at 3:32 @Mr.Wizard, I noticed when I was preparing to post the answer that there seemed to be some oddness around your 1*^4 example as well. I didn't investigate in great detail what's going on here, but I'm thinking there might be a minor bug here. As far as spaces are concerned, there are a few canonicalizations the FE is allowed to make before sending boxes to the kernel. Stripping non-semantic spaces and non-semantic StyleBoxes are among those changes. That the FE does this makes it much easier to write robust MakeExpression rules in the kernel. – John Fultz Oct 20 '12 at 15:47 That makes sense. Is there any way to use InputField to get the version that is sent to the Kernel or$PreRead? – Mr.Wizard Oct 20 '12 at 16:43
Ah...well, it turns out there's a completely different way to do this which additionally allows you to strip (or not) boxes, so long as you can formulate the input as a String. That kind of sounds like what you want. I'll edit the answer to reflect both solutions. – John Fultz Oct 20 '12 at 22:12
On second thought, I think I'll just add a whole new answer. It really is a totally different approach. – John Fultz Oct 20 '12 at 22:19

One approach that may be familiar to more experienced users is based on the input syntax described in String Representation of Boxes.

$$input$$             raw boxes

This appears to work but closer inspection shows that it is not exact:

"$${1*^4, 000123, a*b c}$$" // ToExpression

RowBox[{"{",RowBox[{"1*^4",",","000123",",",RowBox[{"a","*","b","c"}]}],"}"}]

RowBox[{"{",RowBox[{"1*^4",",","000123",",",RowBox[{"a","*","b"," ","c"}]}],"}"}]


(Copy & paste from Show Expression on the lower line.)
Greater changes are apparent in the invalid-expression example:

"$$0001+1,*^6*a b$$" // ToExpression

RowBox[{RowBox[{"0001","+",RowBox[{"","1"}]}],",",RowBox[{"*",RowBox[{"^","6"}],"*","a","b"}]}]

RowBox[{RowBox[{"0001","+","1"}],",",RowBox[{"*",RowBox[{"^","6"}],"*","a"," ","b"}]}]


The other approach I know it to use $PreRead to capture the Box form input: $PreRead = # /. {
RowBox[{"raw", "[", raw__, "]"}] |
RowBox[{raw__, "//", "raw"}] |
RowBox[{"raw", "@", raw__}] :>
ToString[Unevaluated[raw], StandardForm]
} &;


Output still does not precisely match the Show Expression form but it is close:

raw[0001+1,*^6*a b]

RowBox[{RowBox[{"0001","+","1"}],",",RowBox[{"*",RowBox[{"^","6"}],"*","a","b"}]}]

RowBox[{RowBox[{"0001","+","1"}],",",RowBox[{"*",RowBox[{"^","6"}],"*","a"," ","b"}]}]


The discrepancy actually ends up being advantageous as this is the form that is seen by CellEvaluationFunction and that was my original intent.

It is also more convenient than having to wrap input in ToExpression[""]`.

I wonder if there is not a FrontEnd function that already does this.

-