# Intelligent text search on large directories of notebooks?

I have a (deeply nested) directory containing 10s of thousands of wolfram files (mostly notebooks (nbs), but also scripts (.wls), packages (.m, .q wl), my and wxf data) and many subfolders each with a few hundred notebooks on average. Each notebook contains 10-100 of pages. I want to search on text in the files and within relevant notebook cells types e.g. Code, Text, Title, SubTitle, Item, Program, ExternalLanguage, etc.

I'd like to have a live search interface to quickly search through the contents of all files and visually show the matches highlighted within their context.

Is there an existing project or best practices for doing this?

The TextSeach related wolfram symbols are old and seem slow/weak, can mma even do this? Mma is obviously needed to preview the content of matches within notebooks, but what other tools would be used to build such a live deep search?

• Excellent question. Can anything outside MMA search sets of notebooks, for a "specific variable" for instance? This is a tool/question, I'd like answered too. Jun 11 '21 at 5:00
• What exactly is the issue with the TextSearch tools? They are neither particularly old (introduced in version 10.2), nor do the seem too slow after the initial index creation from my quick testing. (To be fair, I did not test it on such huge numbers of notebooks). Jun 12 '21 at 19:08
• TextSearch works for text files but not too well on many large notebooks with lots of graphics. Have you tried with a few long notebooks? I'm thinking about preprocessing all of them and exproting the cell contents to txt files
– M.R.
Jun 14 '21 at 23:24
• If you only want to find cell types you can always grep as a pre-filter (fast) and then use Mathematica tools as the secondary step (slow). You could also use grep to build a token list for cell types and use that as an index. A similar thing could be done to get simple patterns like the Cell["*", "Section" which you could use to build a tag list of sections/subsections in notebooks that you can then do a more sophisticated on-demand filter of by actually opening and preprocessing the content Jun 14 '21 at 23:30
• @bRost03 Right? But that's what is needed. For something like (2) check out resources.wolframcloud.com/FunctionRepository/resources/…
– M.R.
Jun 24 '21 at 19:52