21
$\begingroup$

I am trying to create a Mathematica/shell syntax hybrid. I would like the following command

wget -qO- "http://google.com/space line
break\"" $var

to be interpreted as the following.

wget["-qO-","http://google.com/line
break\"",var]

Notice there are both line breaks and escaped quotes in the command.

I will add a bounty soon. I'm not the best at Regular Expressions but will try it myself.

$\endgroup$
  • 2
    $\begingroup$ How is the "command" entered and stored as a Mathematica object? Is it in a FormBox or plain String or...? $\endgroup$ – Jens May 9 '16 at 5:05
  • 5
    $\begingroup$ "Downvoters/closers please leave comments." How about, what have you tried? You can't protect yourself against this criticism by saying that you haven't tried anything. The last paragraph gives me the feeling that you know that it's not a good question, and you are making excuses for it instead of fixing it. (No downvote from me though.) $\endgroup$ – C. E. May 9 '16 at 5:14
  • $\begingroup$ @Jens plain string. $\endgroup$ – William May 11 '16 at 0:38
  • $\begingroup$ @William Would something similar to my answer of "Programmability of the Natural Language Interface" work for you? $\endgroup$ – Anton Antonov May 11 '16 at 20:18
  • 1
    $\begingroup$ @AntonAntonov The proper way to do this would be to use something like ANTLR which I believe is similar to what you have posted. I am kinda hopeful there is an easier way. I believe I could make the above work if it wasn't for the escaped quotes that make the regular expression rather difficult. I think look behinds might work although I am not good at regular expressions. Would you like me to vote reopen that question? $\endgroup$ – William May 11 '16 at 20:37
21
+250
$\begingroup$

First of all, I agree, as OP mentioned in his comment, ANTLR is one of the proper ways to go.

Now for this specific task, it might be easier to just compose a parser in the "dirty" way, except we don't have to go so far to regex. In my opinion Mathematica's StringExpression is much more powerful and very suitable for the job.

All we have to do is (as OP already did in his answer) to write a parser for CellEvaluationFunction.

However one thing should be take care of when creating the Cell:

The shell emulator is going to process customized commands, which are unlikely to be similar to Mathematica's syntax, so the created Cell should not involve any of the Box typesetting system. One way to ensure this is to create the Cell as Cell[ "", ... ] instead of Cell[ BoxData[""], ... ] (just like what happens when you convert a Cell to the "Raw InputForm"). That case the CellEvaluationFunction receives input as a raw string rather than Boxes, so we can avoid lots of unnecessary work.

One simple parser could be like the following:

Clear[shellEmu`interpretRules, shellEmu`stringMismatchMsg, shellEmu`parser]

shellEmu`interpretRules = {
            str_String /; 
                    StringMatchQ[str, "$" ~~ LetterCharacter ~~ WordCharacter ...] :> ToExpression[StringDrop[str, 1]],
            str_String /;
                    StringMatchQ[str, NumberString]                                :> ToExpression[str]
                          };

shellEmu`stringMismatchMsg[strMarkerPos_] :=
    Failure["OddQuotations", <|
            "MessageTemplate"   -> StringTemplate["Number of quotation markers should be even and positive rather than `Number`"],
            "MessageParameters" -> <|"Number" -> Length[strMarkerPos]|>
                             |>]

shellEmu`parser[inStr_String, form_] :=
    Module[{
               workStr = StringJoin["< ", inStr, " >"],
               strLen,
               strMarkerPos,
               strMarkerPosPart, nonstrMarkerPosPart,
               posMergFunc = Function[pos, Interval @@ pos // List @@ # &],
               res
           },
           strLen           = StringLength@workStr;
           strMarkerPos     = StringPosition[workStr, #][[;; , -1]] & /@ {"\"", "\\\""} // Complement @@ # &;
           strMarkerPosPart = Partition[strMarkerPos, 2, 2, {1, 1}, {}];
           If[Length[strMarkerPos] != 0 && Length[strMarkerPosPart[[-1]]] != 2,
                   Return@shellEmu`stringMismatchMsg[strMarkerPos]
             ];
           strMarkerPosPart    = strMarkerPosPart // posMergFunc;
           nonstrMarkerPosPart = {0, Sequence @@ strMarkerPosPart, strLen + 1} // Flatten // (Partition[#, 2] + {1, -1}) & // Transpose // posMergFunc;
           res = StringTake[workStr, #] & /@ {nonstrMarkerPosPart, strMarkerPosPart};
           res //
               RightComposition[
                   MapAt[ToExpression /@ # &, #, 2] &,
                   MapAt[StringSplit[#, Whitespace] & /@ # &, #, 1] &,
                   Riffle @@ # &,
                   Flatten, #[[2 ;; -2]] &,
                   MapAt[ToExpression, #, 1] &,
                   # /. shellEmu`interpretRules &,
                   #[[1]] @@ Rest[#] &
                   ]
          ]

With that we can now create our ShellEmu Cell as following:

Cell["", "ShellEmu",
        Evaluatable -> True,
        CellEvaluationFunction -> shellEmu`parser,

        Background -> Hue[0.09, 0.41, 0.33],

        CellMargins -> {{50, 0}, {0, 0}},
        CellFrame -> {{False, False}, {5, 5}},
        CellFrameMargins -> {{10, 10}, {10, 10}},
        CellFrameColor -> Hue[0.09, 0.41, 0.56],

        FontFamily -> "Consolas",
        FontColor -> GrayLevel[0.95],
        FontWeight -> Bold,
        Hyphenation -> False,
        FrontEnd`AutoQuoteCharacters -> {},
        FrontEnd`PasteAutoQuoteCharacters -> {}
        ] // CellPrint

ShellEmu 1

Note for a clearer evidence that the command has been correctly parsed, I manually copied the output as an "Input"-style Cell with syntax highlight.

And mismatch of quotation markers will cause a failure:

ShellEmu failure

Now obviously CellPrint is too cumbersome, so we're going to get the job done more automatically.

By defining a new style in the stylesheet as following

Cell[StyleData["ShellEmu"],
      CellFrame                -> {{False, False}, {5, 5}},
      CellMargins              -> {{50, 0}, {0, 0}},
      Evaluatable              -> True,
      CellEvaluationFunction   -> shellEmu`parser,
      GeneratedCell            -> True,
      CellAutoOverwrite        -> True,
      CellFrameMargins         -> {{10, 10}, {10, 10}},
      CellFrameColor           -> Hue[0.09, 0.41, 0.56],
      Hyphenation              -> False,
      AutoQuoteCharacters      -> {},
      PasteAutoQuoteCharacters -> {},
      MenuCommandKey           -> "L",
      FontFamily               -> "Consolas",
      FontWeight               -> Bold,
      FontColor                -> GrayLevel[0.95],
      Background               -> Hue[0.09, 0.41, 0.33]]

we should be able to create "ShellEmu" Cell simply by a shortcut Alt+L.

We can define functions matching the parsed result, say:

Clear[plot]
plot[f_, var_, "from", start_, "to", end_] :=
    Module[{interm},
        Echo[Inactive[plot][f, var, "from", start, "to", end] // FullForm];
        interm = 
            Inactive[Plot][ToExpression@f, ToExpression /@ {var, start, end}];
        Echo[interm // FullForm];
        Activate[interm]
        ]

ShellEmu 2

$\endgroup$
  • 1
    $\begingroup$ Impressive work! $\endgroup$ – Anton Antonov May 13 '16 at 12:19
  • $\begingroup$ @AntonAntonov Thanks! I love your answer under Programming of a natural language interface! $\endgroup$ – Silvia May 14 '16 at 0:44
  • $\begingroup$ Is there any way to get piping to work in your example like something like this. command test | command2 or maybe command test // command2 to be more inline with Mathematica . $\endgroup$ – William May 15 '16 at 21:53
  • $\begingroup$ So I will be giving you the bounty if there are no other answers. I have determined a way to get piping to work in my answer //. $\endgroup$ – William May 18 '16 at 0:34
  • 1
    $\begingroup$ @Liam I believe it will be very hard if not impossible, especially considering that EventKey seems not able to capture Tab. One way might be retrieving keystroke information from OS API. $\endgroup$ – Silvia Oct 28 '16 at 16:09
9
$\begingroup$

I extended the original question to support piping like the following.

wget -qO- "http://google.com" // cat

and it outputs the following.

cat[][wget[-qO-,http://google.com][]]

To get the following.

CellPrint@
  Cell[BoxData[""], "Input", Evaluatable -> True, 
   CellEvaluationFunction -> 
    Function[
     Module[{t}, 
      t = List@
        ReplaceRepeated[NotebookRead[EvaluationCell[]][[1, 1]], 
         RowBox[{x__}] :> x];
      t = 
       Map[Module[{t = #, a, addbacksemicolon = ""}, 
          If[t[[-1]] == ";", addbacksemicolon = ";";
           t = Delete[t, -1]];

          If[Length@t >= 2 && 
            StringMatchQ[t[[1]], 
             RegularExpression["([a-zA-Z0-9])*"]] && t[[2]] == " ", 
           t = SplitBy[t, " " == # &] // DeleteCases@{" "};
           t = SplitBy[t, {"//"} == # &] // DeleteCases@{{"//"}};

           t = Map[
             Map[If[(Characters[#])[[1, 1]] == "\"", #, 
                 StringJoin[{"\"", #, "\""}]] &, #] &, t];

           t = Map[
             StringJoin[StringTrim[#[[1]], "\""], "[", 
               StringJoin@Riffle[#[[2 ;;]], ","], "]"] &, t];
           (*Print["beforeRiffleStringJoin",t];*)

           t[[1]] = t[[1]] <> "[]";

           t = StringJoin@Riffle[t, " // "] <> "" <> 
             addbacksemicolon;
           (* Print@t;*)
           t, t]] &,
        t = (SplitBy[If[Depth[t] == 2, t, t[[1]]], 
            "\[IndentingNewLine]" == # &] // 
           DeleteCases@{"\[IndentingNewLine]"});
        t
        ];
      t = ToExpression@StringJoin@Riffle[t, "\[IndentingNewLine]"];
      t
      ]]];
cd[x_] := Function[SetDirectory[x];]
fn = Function[Null, 
   ReplacePart[
    Function @@ 
     Hold[Null, Quiet[## &, {Function::slotn}], HoldAll], {2, 1, 0} ->
      Function @@ Hold[##]], HoldAll];
cmds = {echo, cat, ls, wget, grep, ps, top, df, killv, rm, cp, mv, 
   cat, mount, chmod, chown, passwd, mkdir, ifconfig, uname, whereis, 
   whatis, locate, find, sed, awk, diff , sort, xargs, uname, 
   ifconfig, locate , man, tail, less, ping, date};
Map[Function[{f},
   With[{n = ToString[Unevaluated[f]]},
    f[x___ : String] := fn[
       Import[
        "!LD_LIBRARY_PATH= " <> 
         If[ToString@#1 == "#1", "", "echo \"" <> #1 <> "\" | "] <> 
         n <> " " <> StringJoin@Riffle[{x}, " "], "Text"]
       ];
    ]
   ], cmds];
$\endgroup$
  • $\begingroup$ Cool! I've played with it a little bit and it's probably better to replace the last Print with CellPrint[ ExpressionCell[ Symbol[StringTrim[t[[1]], "\""]] @@ ToString /@ t[[2 ;;]], "Output"]] $\endgroup$ – swish May 12 '16 at 23:02
  • $\begingroup$ @swish thank you for the upvote(if it was you)! Your code does appear to be shorter I will mess with it a bit. I am trying to figure out if I can generalize the approach above to work for more then just one line. $\endgroup$ – William May 13 '16 at 2:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.