Searching a phrase in all *.nb files

Question

I have so many *.nb files in a folder. In one of them I have written a phrase, unfortunately I cannot remember that file. I am searching about it and the file in which I have written. Is there any way to search to Find a proposed phrase or word? I have used Find but it just work in an opened file not all files in the folder!

Answer 1

Here is a way to search from within mathematica:

notebooks = Quiet@FileNames["*.nb", NotebookDirectory[], 2];
Monitor[Select[
   Table[{nb, 
     StringJoin@Select[ StringSplit[Import[nb, "Plaintext"], "\n"] ,
        ((If[#, Print["match on:", nb]]; #) &@
             StringMatchQ[#, "*NIntegrate*"]) &, 5]},
                {nb,notebooks}], #[[2]] != "" &], {nb}] // Grid[#, Alignment -> {Left, Top}, Dividers -> All] &

This is painfully slow, but it does just search and show only the plain text of the notebook.

Answer 2

A couple of additional solutions. The first, with FindList, is probably the simplest and quickest.

Using FindList

searchDir = "<NB dir>";

fnames = FileNames["*.nb", searchDir, 2];
Length@fnames

sres = {#, FindList[#, {"curve"}, WordSearch -> False]} & /@ fnames;
sres = Select[sres, Length[#[[2]]] > 0 &];
Grid[sres, Dividers -> All, Alignment -> {Left, Top}]

See the options of FindList.

Using CreateSearchIndex

searchDir = "<NB dir>";

index = CreateSearchIndex[searchDir]

sobjs = TextSearch[index, {"curve", "regression"}]

sres = MapThread[{#1, 
     StringCases[#2, "curve" ~~ (Except["\n"] ...) ~~ "regression", 
      IgnoreCase -> True]} &,
   {Through[sobjs["Location"]], Through[sobjs["Plaintext"]]}];

Grid[sres, Dividers -> All, Alignment -> {Left, Top}]

See the signature of TextSearch -- it allows complicated "and", "or", "except" searches.

This solution seems to be fairly slow.

Answer 3

Note: the following method isn't robust. See this answer of mine for a robust solution.

---

Here is an approach which does not rely on the NBImport.exe (which actually performs importing of the NB files as "Plaintext" under the hood) and performs all the operations in the Kernel only. Currently NBImport.exe contains a bug due to which it returns $Failed when have to import a NB file with non-ASCII file path.

The weak side of the following method is that it relies upon the ability of MakeExpression to convert a low-level Notebook expression into the high-level DocumentNotebook what it doesn't always able to do even for correct NB files (and this ability is not guaranteed by the developers in general). This conversion is necessary because ToString doesn't accept raw boxes as the low-level representation of a WL expression (even wrapping the raw boxes by RawBoxes is simply ignored).

The simple function presented below currently fails in many situations but demonstrates the idea.

Here is a function which Gets the contents of a NB file as Notebook expression, then extracts all the Cells as the actual WL expressions wrapped by HoldComplete, converts them into strings and checks whether they contain specified string pattern or not:

findInNBFile[NBFilePath_String, stringPattern_] := 
  Module[{expr = MakeExpression[Get[NBFilePath], StandardForm], cellExprPos, foundPos},
   cellExprPos = Replace[Position[expr, ExpressionCell | TextCell], 0 -> 1, {2}];
   foundPos = 
    Flatten@Position[
      StringFreeQ[
       StringTake[ToString /@ Extract[expr, cellExprPos, HoldComplete], {14, -2}], 
       stringPattern], False];
   If[foundPos =!= {}, 
    Grid[Join[{{Row[{"Found \"", stringPattern, "\" in file \"", NBFilePath, "\""}], 
        SpanFromLeft}, {"Cell #", "The Cell"}}, 
      Transpose[{foundPos, Extract[expr, Most /@ cellExprPos[[foundPos]], HoldForm]}]], 
     Frame -> All], {NBFilePath, False}]
   ];

It can be used as follows:

findInNBFile["ExampleData/document.nb", "abcde"]

Answer 4

Here is a solution written in the spirit of the answer by george2079 but NOT relying upon the buggy NBImport.exe executable. Instead it uses FrontEnd for converting Notebooks into plain text using the findings described here:

findInNBFile::cntconv = "Failed to extract plain text from `1`";
findInNBFile[NBFilePath_String, stringPattern_, resPerFile_Integer: 5] := 
 Module[{NB, NBText, NBlines, matches},
  NB = Get[NBFilePath];
  NB = Notebook[First@NB, 
    Join[FilterRules[List @@ Rest[NB], 
      Except[DynamicUpdating | NotebookDynamicExpression]], {DynamicUpdating -> False, 
       NotebookDynamicExpression -> Null}]];
  NBText = FrontEndExecute[FrontEnd`ExportPacket[NB, "PlainText"]];
  If[TrueQ[Head[NBText] === List && Head[First@NBText] === String],
   NBlines = StringSplit[First@NBText, "\n"];
   matches = Flatten@Position[StringMatchQ[NBlines, stringPattern], True];
   If[matches =!= {}, Print[Length[matches], " matches on: ", NBFilePath]];
   {NBFilePath, Length[matches], 
    StringJoin@
     If[Length[matches] > resPerFile, NBlines[[Take[matches, resPerFile]]], 
      NBlines[[matches]]]},
   Message[findInNBFile::cntconv, NBFilePath];
   {NBFilePath, $Failed,}]
  ]

For example, let us find all Tutorials containing the word NIntegrate. Listing of the all Tutorials:

notebooks = 
  FileNames["*.nb", 
   FileNameJoin[{$InstallationDirectory, "Documentation", "English", "System",
      "Tutorials"}]];
Length[notebooks]
Total[FileByteCount /@ notebooks]/1024.^2

715

215.841

Searching (I won't post here the output except to the timings):

startTime = AbsoluteTime[];

Monitor[Select[Table[findInNBFile[nb, "*NIntegrate*"], {nb, notebooks}], 
   TrueQ[#[[2]] > 0] &], {nb}] // 
 Grid[#, Alignment -> {Left, Top}, Dividers -> All] &

DateString[AbsoluteTime[] - startTime, {"Hour", ":", "Minute", ":", "Second"}]

"00:08:11"

We have successfully processed 715 Notebooks of total size 215 Mb in 8 minutes. Not so bad...

Note that the above function extracts text also from textual elements inside of output, even from the saved (cached) versions of Dynamic expressions. For example in the "Views.nb" Tutorial Notebook the word "Afghanistan" is present only inside of the cached Dynamic output for the second example under the "SlideView" section. Let us try to find it:

findInNBFile[
 FileNameJoin[{$InstallationDirectory, "Documentation", "English", "System", 
   "Tutorials", "Views.nb"}], "*Afghanistan*"]

2 matches on: C:\Program Files\Wolfram Research\Mathematica\10.4\Documentation\English\System\Tutorials\Views.nb

{"C:\\Program Files\\Wolfram Research\\Mathematica\\10.4\\Documentation\\English\\System\\Tutorials\\Views.nb",
2,
"BoxData[Afghanistan]
BoxData[Afghanistan]
"}

Found!

Answer 5

A Unix style text search works best for me. The following search is a timed search over all the .nb files on a 2017 iMac 4GHz Quad Core i7 with 32 GB. The timed find / grep command is:

find . -name \*.nb -exec grep -iH nintegrate {} \; 71.11s user 3.30s system 97% cpu 1:16.48 total

1:16 wall clock time. The files searched were in a multiple directory structure with 2.9 GBytes of files over 1949 .nb files. The directory tree had 1974 subdirectories.

The Mathematica code above that starts with "notebooks = Quiet" required 3:52 wall clock time (mm:ss) or about 3 times slower.