6
$\begingroup$

Mathematica has the built-in function TextStructure, with the option "DependencyStrings" that provides the dependency strings of a sentence. Unfortunately, this function is incredibly slow.

 Timing@TextStructure["He wrote a book. I read the book he wrote.", 
  "DependencyStrings", PerformanceGoal -> "Speed"]
{0.59375, {"(wrote, 2)((nsubj, (He, 1)), (dobj, (book, 4)((det, (a, \
3)))))", "(read, 2)((nsubj, (I, 1)), (dobj, (book, 4)((det, (the, \
3)), (rcmod, (wrote, 6)((nsubj, (he, 5)))))))"}}

As seen it in the example above, two sentences take 0.6 of a second. Does anyone how to speed it up or is there an alternative, as the aim is to use it on the dataset with 15,000+ observations?

$\endgroup$
13
  • $\begingroup$ Have you tried using NLTK. I think it does dependency analysis. It is written in Python so easy to call from WL. $\endgroup$ Commented Mar 3, 2020 at 18:13
  • $\begingroup$ The aim is to use Wolfram language, as I am aware there are more efficient functions in Python. $\endgroup$
    – Tim B
    Commented Mar 5, 2020 at 12:22
  • $\begingroup$ I would like to use WL for everything too, but I often have to take a more pragmatic "use the right tool for the job" approach. It may be because of WL's excessive resource (CPU / Memory) requirements, missing functionality, bugs, ... $\endgroup$ Commented Mar 5, 2020 at 22:35
  • $\begingroup$ Agreed, but the requirement of my research is only to use WL, unfortunately, my hands are tied. $\endgroup$
    – Tim B
    Commented Mar 5, 2020 at 22:46
  • 1
    $\begingroup$ No worries, I am aware of the fact. But maybe it helps someone else in the future. $\endgroup$
    – Tim B
    Commented Mar 9, 2020 at 12:43

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.