Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Hot answers tagged

37

Symbols are created in the current context during parsing. This should not be a problem in normal circumstances as the symbols are merely "initialized" without values or properties. See these posts for more information: Is it possible to use Begin and End inside a Manipulate? Why doesn't this use of Begin[] work? When does Mathematica create a new Symbol? ...


22

Is it strictly necessary? In most cases, no, but doing so gives us many advantages. Most of those advantages might be worked around with various heuristics. But most importantly, it would have been very difficult to have a robust typesetting system. First, the FrontEnd was always going to need some intermediate representation which is not simply textual. ...


21

First of all, I agree, as OP mentioned in his comment, ANTLR is one of the proper ways to go. Now for this specific task, it might be easier to just compose a parser in the "dirty" way, except we don't have to go so far to regex. In my opinion Mathematica's StringExpression is much more powerful and very suitable for the job. All we have to do is (as OP ...


16

The main points of this answer are that,first, it seems rather difficult to have a fully universal mechanism for option-validation, and second, such a mechanism is not currently available in Mathematica on the language level (meaning automation of complete option-checking, including both the option's name and value). In the particular case in question, ...


14

The Documentation states (emphasis is mine): Pattern (:): s:obj represents the pattern object obj, assigned the name s. Optional (:): p:v is a pattern object that represents an expression of the form p, which, if omitted, should be replaced by v. I think that the Documentation page for Optional is misleading since it states that the infix operator for ...


14

Not really an answer and I may have to delete it, but...I had something of a different reaction to this business of the front end having its own parser, circa 2004. For motivation I recommend the original. I knew a front end that swallowed a kernel. I don't know why it swallowed a kernel. They taste infernal. I knew a front end that swallowed a ...


13

tokenize[str_] := Module[{exp, nb = CreateDocument[{ExpressionCell@ InputForm@MakeExpression[str, StandardForm]}, Visible -> False]}, SelectionMove[nb, Next, Cell]; exp = Flatten[ NotebookRead[nb][[1, 1]] /. {RowBox -> List, i_String /; StringMatchQ[i, Whitespace ..] :> Sequence[]}]; NotebookClose[nb]; exp[[3 ;;-2]]...


13

I am a developer at Wolfram Research and I am trying to share some of the work I have been doing with parsing WL code. I have written a package for parsing WL code and retaining interesting metadata, such as file and line information. I also expose a tokenization function. The paclet is available on the public paclet server: In[1]:= PacletUpdate["AST","...


13

This answer is based on the Mathematica functional parsers package FunctionalParsers.m used and described in these blog posts. This command loads that package: Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/FunctionalParsers.m"] Using direct parser definitions (first answer) Consider the EBNF: <space> = { '...


12

No, AFAIK there is no way to see the FullForm and I think your conclusion is correct. The ^^ is not an operator, it is a form how you can input a number. Effectively, this behavior applies to all form of numerical input. For instance this is unholdable too HoldComplete[16*^2] (* HoldComplete[1600] *) Advanced expanation To give a more thorough ...


12

I worked on Interpreter. As far as the implentation is now, the DelimitedSequence parser does not support quoting, so what you want can't be done. We'll try to add it in a future version.


12

To not mess with $ContextPath you can Block it just for that evaluation. e.g. Block[{$ContextPath = {}}, ...]. But we don't want to do that for the evaluation, input for FullForm should be evaluated freely and only the result should be returned with full symbols' names. To achieve that you should know: $ContextPath and $Context affect the typesetting. ...


11

Here is how I would approach this problem. First, when possible we should make use of the Attributes of the functions in use with regard to their effect on pattern matching. These attributes follow the natural application of the functions to which they apply so often they make mathematical matching easier. (Sometimes you don't want that.) The OneIdentity ...


11

The following code defines the function randomSentence. It uses recursive descent to generate a sentence from a grammar expresssed as rules: ClearAll[randomSentence, randomCount] randomCount[] := RandomVariate[GeometricDistribution[0.5]] SetAttributes[randomSentence, HoldAll] randomSentence[rules_, expr_] := Module[{generate} , SetAttributes[generate, ...


11

I'd do the minimum necessary to make a legitimate, unambiguous Mathematica expression, and then let Mathematica rewrite it. stepexpr[s_] := ToExpression[StringReplace[s, {"(" -> " dummy[", ")" -> "]"}]] /. op_Symbol dummy[args__] -> op[args] /. dummy -> List This replaces () expressions with a dummy[] function in the string, making a legal ...


11

This can be done using functional parsers. Does the solution below look like something you would be interested in? (If you are I will elaborate on the current answer...) Grammar Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/FunctionalParsers.m"] ebnfToy = " <program> = ( [ <wgap> ] &> <...


10

This seems to work: StringCases["blabla ...Hello Hello ... blabla ... Goobye Goobye ..", Longest[___ ~~ a : "Hello"] ~~ b : Shortest[___ ~~ "Goobye"] :> a ~~ b] Update If there are multiple substrings to extract you can use recursion: extractbetween[str_, x_, y_] := Module[{f}, f[s_] := StringCases[s, Longest[a___ ~~ x] ~~ b : Shortest[___ ~~ ...


10

The result of the front-end's parsing is definitely used, but it does not need to be complete, it seems. This can be seen by constructing Cells/Boxes manually. Try RawBoxes@RowBox[{"1", "+", RowBox[{"2", "*", "3"}]}] RawBoxes@RowBox[{RowBox[{"1", "+", "2"}], "*", "3"}] RawBoxes@RowBox[{"1", "+", "2", "*", "3"}] These all display as 1 + 2 * 3 but you can ...


9

Since no full solution has arrived, I've created the code based on the different sources you have provided. The function applyRulesRepeated scans through the initial sentence and replaces the first nonterminal symbol it founds while holding all external functions (f, g, ..., for suffixing or alike). Here I used the holding part of Archimedes' answer. This ...


9

I'm posting this variant in the hopes that it will be a little more educational. Otherwise doesn't add anything over Kuba's version. Generally, parsing can be done using StringCases. You'll need to build up a string expression that describes the pattern of the file name, much the same way you'd write "%d_%s_Polarizer%dDeg-Temp%dK.dpt" when working with ...


9

This is an ideal use case for SemanticImport, but unfortunately it has issues getting the commas right in version 10.0. Luckily, version 10.0.1 has already fixed this bug:


9

I extended the original question to support piping like the following. wget -qO- "http://google.com" // cat and it outputs the following. cat[][wget[-qO-,http://google.com][]] To get the following. CellPrint@ Cell[BoxData[""], "Input", Evaluatable -> True, CellEvaluationFunction -> Function[ Module[{t}, t = List@ ...


8

shortestStringCases[str_String, from_String, to_String] := StringCases[ str, (from ~~ mid___ ~~ to) /; StringFreeQ[mid, {from, to}]] shortestStringCases["blah X blah X first Y blah X blah X second Y", "X", "Y"] (* {"X first Y", "X second Y"} *)


8

The simple answer is, if you want a string converted to StandardForm, you could wrap BoxData around it. E.g., CellPrint[Cell[BoxData["myFunction::usage=\"myFunction does ...\";"], "Input"]] But, in general, I wouldn't structure this as a question of CellPrint vs. FrontEnd`CellPrint. FrontEnd`CellPrint is undocumented, and therefore there is no contract ...


8

Recent versions of Mathematica no longer use Java for parsing JSON, and work very fast. There are at least three ways to read JSON. Import[..., "JSON"] is fast an reliable. It returns nested Lists and Rules. Import[..., "RawJSON"] is faster than "JSON" and returns nested Lists and Associations. These are much easier to work with than Rules. The Developer` ...


8

Just define a new function that could take different inputs, either through overloading or using If, Which, or Switch. Clear[f] f[x_. Power[z, e_.]] := {x, e} f[Times[x_, Power[z, e_.]]] := {x, e} f[x_] := {x, 0} Cases[lis, y : Alternatives[x_., x_. Power[z, e_.], Times[x_, Power[z, e_.]]] :> f[y]] (* {{6, 0}, {1, -4}, {4, -2}, {4, 2}, {1, 4}} *) ...


8

For producing a NB file without line breaks you can Get it as a Notebook expression, then Export it with PageWidth -> Infinity as "Package" (other possible options does not work correctly: Export ignores PageWidth -> Infinity when exporting as "NB" and corrupts the code when exporting as "Text"): Export["document.nb", Get["ExampleData/document.nb"], "...


8

Begin["mycontext`"] affects the parser, not the evaluator. These are separate expressions: Begin["mycontext`"]; myvar = 7; Print[Context[]]; Print[Context[myvar]]; End[]; They get parsed and evaluated one by one. Once Begin["mycontext`"]; is evaluated, the new context applies to all subsequent lines. This is a single expression: Do[ Begin["...


8

Note: If you want this type of parser added for JSON, I think WRI could do it easily. They already have Developer`ReadRawJSONStream so I think adding some event-based parsing to that wouldn't be too much to ask This has been kicking around my for the past few days, so here's a quick example of what I discussed in my comment. It is by no means complete, but ...


7

I think this particular scenario has to do with how you can create your own Import/Export filters: Developing an Import Converter Regarding 'verification' as in the Plot[Sin[x], {x, -Pi, Pi}, Frame -> True, FrameTicksStyle -> Red] example given by Nasser, keep in mind that you might have options parameterized like so: Manipulate[ Plot[Sin[x], {x, -...


Only top voted, non community-wiki answers of a minimum length are eligible