13
$\begingroup$

This code works fine in a string based interface like WolframScript or Mathematica Kernel.

Run this through a string based interface

StringCases["hello", x_..]

Or do 

ToExpression["StringCases[\"hello\", x_..]"]

in the notebook. Both work correctly, x_.. is interpreted as a Repeated form of the pattern x_,

Try it online!

In the notebook based front end, however, it fails because x_. is interpreted as a Default pattern and the second dot throws a syntax error.

enter image description here

The code must be modified to

StringCases["hello", (x_)..]
(*    {"h", "e", "ll", "o"}    *)

Why is there a difference in behavior between the front end and kernel parsers.

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
8
$\begingroup$

I suspect, much to my chagrin, that the FE is right and the InputForm parser is wrong. For one thing, the kernel itself thinks that parens are needed for Repeated:

Repeated[Pattern[x,Blank[]]]//InputForm
(*(x_)..*)

The reason it breaks in the FE is that the FE treats the x_. as a single token: RowBox[{"x_.", "."}]. And that's absolutely correct: see tutorial/OperatorInputForms and the section "Spaces to Avoid", because "x_." is a "special input form". On the other hand, .. is a normal operator, so it is another token. For example a.. is RowBox[{"a", ".."}]. So the question is whether "x_.." should be RowBox[{"x_.","."}] or RowBox[{"x_",".."}]. And what happens if something is added after the second dot? Now, according the precednece tables, forms involving _ have higher precedence than pretty much any operator (MessageName seems to be the lone exception). In which case the FE is doing the right thing. I'll ruminate and discuss some more whether that's the resolution, but this is clearly a bug one way or the other.

P.S. We do take these differences seriously, especially when easily typable input parses differently in the two contexts.

Improve this answer
$\endgroup$
9
  • $\begingroup$ "P.S. We do take these differences seriously, especially when easily typable input parses differently in the two contexts." I have many, many examples. See my links above if you are interested. $\endgroup$
    – Robert Jacobson
    Mar 24, 2020 at 22:26
  • $\begingroup$ Today I collected a few of them here: wltools.github.io/LanguageSpec/Resources/KernelVsFE. I have not included difference in operator precedence on that page. $\endgroup$
    – Robert Jacobson
    Mar 25, 2020 at 4:14
  • 1
    $\begingroup$ I see several errors on that page. Most signifcantly, FullForm doesn't depend on the environment--parsing of an input might be context-dependent, but once you have an expression its FullForm is unambiguous. However, you also point out real issues (many of which are known). For example the Times issue you point out is not, in fact, always, semantically insignificant (for finite precision arithmetic). We actually have a fix for that in our internal prototype build and I'm optimistic it will make into our next major release. I am happy to discuss further in some suitable venue. $\endgroup$
    – Itai Seggev
    Mar 25, 2020 at 7:12
  • 1
    $\begingroup$ @RobertJackson I will note that Itai spent significant time testing and debugging the pending changes to the kernel times/unary minus parsing. $\endgroup$
    – Daniel Lichtblau
    Mar 25, 2020 at 15:04
  • $\begingroup$ "Most signifcantly, FullForm doesn't depend on the environment..." I assume you are referring to differences between systems. I haven't witnessed it—I am just quoting the docs which might be incorrect. Thanks for the correction re: the Times issue. I will need to correct the page. Let me know of any other errors. You're the first person from Wolfram to express any interest in this, as most of it has been known to be broken for years. If you would like more, I can look through my notes. (That website was motivated by differences in operator precedence between notebook and command line.) $\endgroup$
    – Robert Jacobson
    Mar 25, 2020 at 20:08
11
$\begingroup$

The notebook interface and the kernel have different parsers, because the notebook needs to manipulate the input before the kernel sees it for a variety of reasons. There are many differences between the two parsers. It has not been a priority to Wolfram to fix the differences.

I have been trying to catalog the differences here (rendered as a webpage here). Most differences are semantically equivalent, but some are meaningful, as you have discovered.

Improve this answer
$\endgroup$
4
  • 3
    $\begingroup$ I have, with some considerable effort, attempted to spelunk your interesting website for the list of FE/kernel differences, and I've not been able to find any at all. Could you point specifically to where I can find your enumerated list of differences? $\endgroup$
    – John Fultz
    Mar 24, 2020 at 22:28
  • 1
    $\begingroup$ The website is a bit broken. Sorry for that. I'll see if I can fix it today. The easiest place to find differences is in the table of operator precedences. There are also some differences mentioned elsewhere. The site isn't organized for the purpose of pointing out these differences, though. Given that both you and @Itai Seggev have shown interest in the subject, maybe I should write up something and post it in question form here on SE. $\endgroup$
    – Robert Jacobson
    Mar 24, 2020 at 22:36
  • $\begingroup$ @JohnFultz I collected a few of them here: wltools.github.io/LanguageSpec/Resources/KernelVsFE. I have not included difference in operator precedence on that page. $\endgroup$
    – Robert Jacobson
    Mar 25, 2020 at 4:13
  • $\begingroup$ @JohnFultz "I have, with some considerable effort, attempted to spelunk" Perhaps. But that's still not the right way to spell lunk. $\endgroup$
    – Daniel Lichtblau
    Mar 26, 2020 at 0:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.