# how to easily find location of hidden characters that causes hard to find syntax error in notebook?

For last 30 minutes (and I was lucky today, sometimes I spend 5 times as much time) I have been removing code little by little from my large one cell, (have to be one cell, it is Manipulate demo stylesheet) in order to find where the syntax error is (since Mathematica, with all its might and power, can't tell the user which line number the syntax error is located at). Each time I see the red line show up on the right side of the notebook, indicating a syntax error, I know that I will now waste another hour at least looking for the syntax error.

Why is it that hard for Mathematica to tell the user where the syntax error is?

I zoomed down to this little 5 character code, and I copy it to new notebook to examine it:

So, there is a hidden bad character there. Sometimes, with my big fingers and my small keyboard (can't find big keyboards any more, everything is so tiny), I must have hit wrong key combination, which must have injected some bad character to the notebook, and this happens.

I will keep this cell on its own in order to find what the problem is. I tried to select the cell, and do convert to input form, but nothing happened. no output came of it.

Then I did cell->ShowExpression, and this came up

Cell[BoxData[
RowBox[{"(",
RowBox[{"k", "h"}],
SuperscriptBox[")", "2"]}]], "Input",
CellChangeTimes->{3.536893041212217*^9},
EmphasizeSyntaxErrors->True]


So, what does all this mean? Do you see from the above what is the problem?

I think I entered the superscript as I always do, using Ctrl ^ then 2.

I think the notebook interface is powerful and allows many things not possible otherwise, but because of problems such the above, I wish sometimes I am using plain text editor like emacs, where I can see what I have and not worry about hidden things I do not see, but I tried that once, and it is not practical overall setup to work with.

btw, originally, I had lots of code like the above inside strings, i.e. inside " " for formatting, and now I am removing all the strings to do some other way of formatting. I do not know if this has anything to do with it. But as you can see, now there is no string around the expression. Here is a screen shot of the earlier version of the notebook, from the same location of the code, showing how the above was inside a string, and I simply was removing the outside string quotes, that is all:

-
Instead of using strings for displaying equations - it is probably best to use HoldForm. – Simon Jan 30 '12 at 7:04
Btw - there might be a problem in the ToExpression code that your example reveals. ToExpression["(h k)^2"] works fine, but the version with a superscript, $\text{ToExpression["}(h\ k)^2\text{"]}$, produces an error. For reasons very similar to the example that you gave: The string with a superscript is interpretted using the string representation of boxes "(k h\!$$\*SuperscriptBox[\()$$, $$2$$]\)" – Simon Jan 30 '12 at 7:11
Out of curiosity, why do you think all your code has to be in a single cell for demonstrations? (I've split out utility functions into separate cells for demonstrations I've written in the past.) – Brett Champion Jan 31 '12 at 5:07
@BrettChampion, I am not sure how you do that? How do you break a Manipulate into many cells? In the Demonstration style sheet, there is the initialization section, which is one cell, and the Manipulate itself, which is one cell. Which one of your demos you did that? thanks – Nasser Jan 31 '12 at 10:42
btw, I do not use the Initialization cell (at the top of the notebook). Instead, I use the Manipulate cell and then type Initialization:>{....} to add the initialization functions in there. When using the separate initialization cell, one must also add SaveDefinitions->True to Manipulate (it says so in the notebook). This has caused me strange problems when I do that. So that is why just use the Manipulate cell. That works fine and never had problems with that setup. I use the Demonstration stylesheet ofcourse. – Nasser Jan 31 '12 at 10:51

There was actually an old post on mathgroup, Strange Syntax problem, that looks like it was exactly the same problem you're experiencing. A power/superscript being associated with the closing parenthesis instead of the whole parenthesized expression (see the box forms presented in my original answer below for more details). Unfortunately, this problem was not resolved in the the mathgroup post.

The best I can suggest at the moment is to simple press the + button
to get the expanded form

This should help narrow down the problem.

Alternatively, if it is this particular problem, you could Show Expression, then search for the problematic code snippet SuperscriptBox[")". Then you can modify the box structure directly. This is normally easiest if you past it into a new input cell so that the Extend Selection mechanism works.

There is no "hidden bad character" in the cell that you posted. Rather, the box structure is crazy and can't be interpreted by Mathematica - I have trouble imagining how you managed to create it...

Cell[BoxData[
RowBox[{"(",
RowBox[{"k", "h"}],
SuperscriptBox[")", "2"]}]], "Input"]


and here is how it should look

Cell[BoxData[
SuperscriptBox[
RowBox[{"(",
RowBox[{"k", " ", "h"}], ")"}], "2"]], "Input"]


So, your question does not really a work with the example you provided.

However, a simple way to maybe clean your code (at least in this case) is to copy it as plain text or input text, both of which produce (k h)^2 when pasted.

-
 However, a simple way to maybe clean your code (at least in this case) is to copy it as plain text or input text. Thanks for the suggestion. But I can't really do this, I have 10,000 line of code cell. If I copy it to notepad and copy it back to the Mathematica notebook, it will break all sort of formatting I had? I tired such things before in the past, they do not really work for me. The problem is how to find location of bad characters in the code in the first place. Once the location is found, then the solution is easy. – Nasser Jan 30 '12 at 7:38 @Nasser: As I said in my answer, there are no "bad characters" in your code - just bad box structures. – Simon Jan 30 '12 at 7:48 And as I've said to you before, you should divide up you code and move things into initialization cells (as you are allowed to do in demonstrations). 10,000 lines of code is just getting ridiculous! This really would make your life easier, since the line breaking + indenting algorithm of cells becomes slow as they get large. Another option (that you already discarded, but I can't remember why) is to use "Code" cells or to (temporarily) turn off AutoIndent. – Simon Jan 30 '12 at 7:50 Ok, then how to find location of bad box structure? what it is called is not really important for me, what is important, is how to find such location. Regarding the initialization code, it is about 6,000 lines of code. The UI control code is about 4,000 or so. I have both in one cell now. But even if make the initialization code a separate cell, it is still too large. The demo stylesheet does not allow one to break things so that each function is in one cell on its own. I wish that was possible, it will make life much easier. thanks – Nasser Jan 30 '12 at 7:56 Regadring the slowness part when cell gets large. I know about that. But I found if I un-select the Enable automatic syntax coloring from preferences, then the notebook is no longer as slow as before, and this really helped allot, otherwise, I would not be able to do this. When I need to check something, then I enable this back – Nasser Jan 30 '12 at 7:59