17
$\begingroup$

I've just started using the Mathematica natural language interface and found it very interesting. After doing some research, I found some documentation on the "Programmable Linguistic Interface":

https://reference.wolfram.com/language/guide/ProgrammableLinguisticInterface.html

After a quick read through the docs what I couldn't find was if you could actually extend the 'main' natural language grammar or program the interface to have such extensions. Maybe this is possible by manipulating a global set of grammar rules or using some functions I'm missing.

Building on the natural language interface used with Mathematica would be a great project. Any advice if this is possible?

Here is a concrete example. I'm thinking of something along the lines of being able to say:

"set the value of l to be MSE[4,5,{k,j}]"

so I can then use the commands:

"set mylist to {l}"
"the head of the first element of mylist"

The current idea is to be able to add to the grammar rules of Mathematica's main natural language parser so that the natural language interface would be able to handle new facts, phrases and functionality. Can this be done, or do you have to make a custom parser from scratch to handle the whole new language using the (ample) framework of language datatypes provided by the build in interpreters?

$\endgroup$
  • $\begingroup$ Welcome to Mathematica.SE! I suggest the following: 1) As you receive help, try to give it too, by answering questions in your area of expertise. 2) Take the tour! 3) When you see good questions and answers, vote them up by clicking the gray triangles, because the credibility of the system is based on the reputation gained by users sharing their knowledge. Also, please remember to accept the answer, if any, that solves your problem, by clicking the checkmark sign! $\endgroup$ – Michael E2 May 7 '16 at 1:39
  • 1
    $\begingroup$ This seems a very broad question. This site tends to be for more specific questions. $\endgroup$ – Michael E2 May 7 '16 at 1:44
  • 1
    $\begingroup$ Could you give an example of what you want to do? It's not clear to me. $\endgroup$ – C. E. May 7 '16 at 5:13
  • 3
    $\begingroup$ I'm voting to close this question as off-topic because the OP is asking for a capability that does not exist in any version of Mathematica released so far. $\endgroup$ – m_goldberg May 10 '16 at 16:50
  • 1
    $\begingroup$ The head in the usual sense is not the 1st part, but the 0th. This would make the "head of the head of the list" Symbol, not 1. While I think that what you ask for is impossible, these confusing instructions could make it doubly so! $\endgroup$ – Oleksandr R. May 11 '16 at 3:12
25
$\begingroup$

OP seems to be mostly interested in Mathematica's built-in capabilities for grammar definition, parsing, and interpretation, but I think some of the questions asked can be seen and answered within a more general development perspective.

General

Building on the natural language interface used with Mathematica would be a great project. Any advice if this is possible?

This can be done using functional parsers. See these blog posts with detailed examples:

  1. "Natural language processing with functional parsers",

  2. "Simple time series conversational engine".

Here is a general answer / discussion on creating Domain Specific Languages (DSLs):

Extending and maintaining grammars

The idea is to be able to add to the grammar rules of the main natural language parser so that the natural language interface can be extended with new facts, phrases and functionality. Can this be done, or do you have to make a custom parser from scratch to handle the whole new language using the (ample) framework of language datatypes provided by the build in interpreters?

The package "FunctionalParsers.m" can produce parsers from Extended Backus-Naur Form (EBNF) of the grammars. With that package designing grammars and adding new rules to grammars becomes a much easier task.

(Three years ago I worked with the Nuance Speech Recognition System and I was feeding that system with grammars derived from text corpuses and data bases using Mathematica.)

On the concrete example

Similar to the concrete example in the question:

"set mylist to {l}"
"the head of the first element of mylist"

the time series conversational engine can work with the following sequence of commands:

"load data file '~/example.csv'"
"least squares fit with x+Sin[x]"
"find bottom outliers"

(See this movie also linked in the mentioned blog post.)

Code

Using the concrete example in the question I programmed the following parsing and interpretation. I did not try to be as complete as possible, just to provide a good enough example.

Load the package:

Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/FunctionalParsers.m"]

EBNF

Here is a grammar that reflects the concrete example:

ebnfCode = "
  <command> = <assignment> | <position-query> ;
  <assignment> = ( 'set' &> <var-name> , ( 'to' | 'as' ) &> <value> ) | ( 'assign' &> <value> , 'to' &> <var-name> ) <@ Assign ;
  <var-name> = '_WordString' <@ Var ;
  <value> = '_String' <@ Value ; 
  <position-query> = ( 'element' &> <pos-int> | [ 'the' ] &> <pos-word> <& 'element' | <pos-head> ) , 'of' &> ( <position-query> | <var-name> ) <@ PositionQuery ;
  <pos-int> = 'Range[1,100]' <@ PosInt ; 
  <pos-word> = 'first' | 'second' | 'third' | 'tenth' <@ PosWord ;
  <pos-head> = [ 'the' ] &> 'head'  <@ PosInt@0& ;
  ";

Note the recursive definition of the <position-query> rule.

Parser generation

Generate the parsers from EBNF code.

GenerateParsersFromEBNF[ParseToEBNFTokens[ebnfCode]];

As mentioned above with this function we can easily extend existing grammars.

Verification

statements = { "assign {3,4,{4,a},5,k} to mylist", 
   "element 4 of mylist", "third element of mylist", 
   "head of mylist" , "second element of element 3 of mylist"};
ParsingTestTable[pCOMMAND, statements, "Layout" -> "Vertical"]

enter image description here

Interpretation

At this point we write interpreters of the parsing output.

Block[{Assign, PositionQuery, PosWord, wordToIntRules},
 wordToIntRules = 
  Thread[{"first", "second", "third", "tenth"} -> {1, 2, 3, 10}];
 Assign[parsed_] :=
  Block[{varName, value},
   varName = First[Cases[parsed, Var[v_] :> v, Infinity]];
   ClearAll[Evaluate@varName];
   value = 
    ToExpression@First@Cases[parsed, Value[v_] :> v, Infinity];
   With[{sn = ToExpression@varName, v = value}, 
    OwnValues[sn] = {HoldPattern[sn] :> v}]
   ];
 PositionQuery[parsedArg_] :=
  Block[{parsed = parsedArg},
   parsed = parsed /. PosWord[pw_] :> PosInt[pw /. wordToIntRules];
   If[Length[Cases[parsed, Var[v_] :> v, Infinity]] > 0,
    Part[
     ToExpression@First@Cases[parsed, Var[v_] :> v, Infinity],
     ToExpression@First@Cases[parsed, PosInt[p_] :> p, Infinity]],
    Part[
     First@DeleteCases[parsed, _PosInt],
     ToExpression@First@Cases[parsed, PosInt[p_] :> p, Infinity]
     ]
    ]
   ];
 statements = { "assign {3,4,{4,a},5,k} to mylist", 
   "element 4 of mylist", "third element of mylist", 
   "head of mylist" , "second element of element 3 of mylist"};
 ParsingTestTable[pCOMMAND, statements, "Layout" -> "Vertical"]
]

enter image description here

$\endgroup$
  • $\begingroup$ Great answer. Thank you. $\endgroup$ – Burton Samograd Jan 10 '18 at 5:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.