Managing formatted usage messages in Wolfram Workbench

Question

We have numerous packages (defined in .m files) that we maintain and develop in WolframWorkbench (2.0), and in those packages, we use the traditional method of determining which symbols are public by assiging ::usage messages to them. It's also nice to have at least that much documentation for every public symbol. I've been spending some time reviewing and improving the documentation for these public symbols, and one thing I'd like to be able to do is emulate the formatting that Wolfram uses for its own usage messages, for example:

It's not particularly difficult to make that sort of formatted text in the notebook interface. However, getting that formatted text into a package file in Workbench requires copying the formatted text, pasting it into an input cell, and then copying the resulting string into Workbench:

"\!\(\*RowBox[{\"Binomial\", \"[\", RowBox[{StyleBox[\"n\", \"TI\"], \
\",\", StyleBox[\"m\", \"TI\"]}], \"]\"}]\) gives the binomial \
coefficient \!\(\*RowBox[{\"(\", GridBox[{{StyleBox[\"n\", \"TI\"]}, \
{StyleBox[\"m\", \"TI\"]}}], \")\"}]\). "    

The additional copy/paste step is requiried to keep some of the quotes escaped; it goes wrong if you paste directly to Workbench.

Suffice it to say, this is a pain, and editing the resulting string directly is pretty unpleasant. So far the best I've been able to figure out for editing the strings is to do something like this in a notebook file:

CellPrint[Cell[DisplayForm@Binomial::usage, Editable -> True]]

Is there a better way to do this? Just having the arguments in usage strings appear in italics is a big win from a readability standpoint.

UPDATE in respose to @István Zachar's comment: I knew about the "Show Cell Expression" menu item mentioned in the answers to your question, but I didn't realize that you could edit the displayed cell expression. That's less awkward than the CellPrint approach, at least.

UPDATE the second: The main reason we don't just use autogenerated packages is that we have a setup where "build" an application from packages for deployment on remote machines, and all the packages are in Subversion. Using autogenerated packages with this workflow is very awkward.

Answer 1

This problem exists because WB encourages you to edit the .m file directly. If you created and edited an .nb (package) file -which automatically creates and updates a .m file - this problem (and others) would not exist. Indeed, if you work with the .nb (package) file you have all the cell organizational/styling facilities available. So when I recently moved to WB, I wanted WB for creating documentation, Munit testing, ect...TOGETHER WITH front end .nb editing capabilities. Is this possible? Fortunately yes. Here is how (very detailed for beginners):

If you have a xyz.m file, create next to it (ie. in the SAME folder, which is inside the WB workspaces folder) a package file xyz.nb (you create this file in MMA or even in WB, I think). WB will- by default - load MMA front end whenever you want to edit this .nb file. You put in this .nb file all the code you may already have written in the .m file. You may leave the code in Code cells, but (much) preferably you put it in Input-Initialization cells. Then you save the .nb file. This will automatically erase and update your old .m file (so back it up - BEFOREHAND - the first time, just in case). Now you have a .nb file next to a synchronized .m file. Next time you want to edit your code, double-click the .nb file in WB. This will automatically load MMA front end and update the .m file when saving the .nb file. If you want to debug, just use the .m file in WB, but remember that any changes you make should in the end be made to the .nb, if you want to keep them. So we can have the best of WB and MMA together.

Advantages of .nb (package) editing:

• nicely hierarchically formatted .nb package file
• nicely formatted usage messages
• syntax highlighting
• possibility to add text cells for comments, explanation, ect

HTH

Answer 2

This is a solution to the problem that seems well-suited to our workflow, which draws on Albert Retey's answer and the answers to @István Zachar's question, but I may decide I hate it after I become more familiar with it. Its main drawbacks are that it requires a separate notebook file that you keep alongside the package file and (potentially) introduces another dependency. However, it doesn't require any copying and pasting or doing round-trips through "Show Cell Expression", which after brief experimentation I've found to be error-prone and liable to crash the Mathematica front-end.

First, you make a text cell in the notebook, and you enter whatever fancily-formated usage message you want. For example:

This will create a cell expression that's almost exactly what you want to convert into a usage string using the procedure described in Albert Retey's answer:

Cell[TextData[{
 "Wibble[",
 Cell[BoxData[
   FormBox["n", TraditionalForm]]],
  ", ",
 Cell[BoxData[
  FormBox["m", TraditionalForm]]],
 "] quuxualizes everything."
}], "Text",
 CellChangeTimes->{{3.5427144059586124`*^9, 3.54271444018589*^9}}]

The one problem is that DisplayForm only works properly with box expressoins, and the contents of TextData don't count. There are a few ways of handling this, but the best I've involves selecting the text cell and selecting the Convert To > Standard Form Display menu item. Doing that turns the cell contents into boxes.

Now we need a way to get the cell expression to the kernel. We can do this by using the cell tags feature, which is accessible on Windows using CTRL+J. Then add a tag that seems appropriate; I use "Wibble::usage" in this case. Finally, save the notebook. You'll have something that looks like this:

Now in your package file, you can get everything you need from the notebook, and convert it into a list of rules associating usage strings with symbol names, using Get and Cases:

usageForms = Cases[Get@"UsageStrings.nb",
  Cell[BoxData[stuff_], ___, CellTags -> tag_, ___] /;
    StringMatchQ[tag, __ ~~ "::usage"] :>
   (StringReplace[tag, "::usage" -> ""] -> 
     ToString[DisplayForm@stuff, StandardForm]),
  Infinity];

You can use this for as many usage strings as you like, and because it only picks up cells that have CellTags of the form *::usage you can have any other convenient documentation in the notebook as well. Section headings, text, graphics, it's all fine. From there, it's pretty simple to assign the usage strings to their affiliated symbols, but I went ahead and packaged it up as a function:

AssignUsage::nousg = "No usage message in `1` for symbol `2`";

Attributes[AssignUsage] = {HoldFirst};

AssignUsage[s_Symbol, rules : {___Rule}] :=
 With[{name = SymbolName@Unevaluated[s]},
  If[FreeQ[rules, (name -> _)],
   Message[AssignUsage::nousg, rules, HoldForm[s]];
   $Failed,
   MessageName[s, "usage"] = name /. rules]]

Then instead of just assigning the message directly, you use:

AssignUsage[Wibble, usageForms]

Now Wibble's usage information comes through in all its beautifully formatted glory:

Answer 3

Another approach that might suit you (although you've probably settled on one by now) is to use Workbench's built-in Documentation Tools to create documentation for each symbol, and then automatically export formatted usage messages from these documentation notebooks to a .m file.

The Documentation Tools provide a way to export the usage messages via the function CreateTextUsageMessageFile in the sub-package DocumentationBuild`SystemResourcesFromDocs`, but unfortunately this removes any formatting.

Instead, we can achieve this with the following bit of code:

ProcessDirectory::usage = 
    "ProcessDirectory[inputDir, outputFile] parses the usage messages for all \
documentation notebooks in inputDir and writes the output to outputFile.";
ProcessSourceNB::usage =
    "ProcessSourceNB[nbFile_, outputFile] parses the usage messages for the given \
documentation notebook and appends the output to outputFile.";

ProcessDirectory[directory_String, outputFile_String, log_:Print] :=
    Module[
        {
            files = FileNames["*.nb", directory]
        },
        log["Found " <> ToString@Length@files <> " source notebooks."];
        Put[outputFile];
        log["Cleared output file " <> outputFile];
        ProcessSourceNB[#, outputFile, log]& /@ files;
    ];

ProcessSourceNB[sourceNB_, outputFile_, log_:Print] := 
    WriteUsageMessage[
        ParseUsageMessage@GetUsageMessage@sourceNB,
        outputFile,
        log
    ];

WriteUsageMessage[{symbol_,usagem_}, outputFile_, log_:Print] :=
    With[
        {
            mess = StringTrim@ConvertToString@usagem,
            symb = ToExpression@symbol
        },
        PutAppend[
            Unevaluated[symb::usage = mess;],
            outputFile
        ];
        log["Wrote usage message for " <> symbol];
    ];    
WriteUsageMessage[{nbName_,Null}, _, log_:Print] :=
    log["Skipped notebook " <> nbName];

(* Returns {symbolName, usageMessage}, or {symbolName, Null} if nothing was found. *)
GetUsageMessage[sourceNB_] := 
    Module[
        {
            nbfile = Get[sourceNB],
            objectName, usages
        },
        (* Get the name of the symbol, and the usage cell. *)
        objectName = Cases[
            nbfile, Cell[name_,  "ObjectName", ___] :> name, {0,Infinity}, Heads->True
        ];
        usages = Cases[
            nbfile, Cell[usage_, "Usage", ___] :> usage, {0,Infinity}, Heads->True
        ];

        (* Return *)
        If[ Length /@ {objectName, usages} =!= {1,1},
            {Last@FileNameSplit[sourceNB], Null},
            First /@ {objectName, usages}
        ]
    ];

Calling ProcessDirectory[inputDir, outputFile] (either manually or from an Ant build task) writes formatted usage messages once and for all to an output file you can include in your package distribution.

I haven't included the function ParseUsageMessage (which actually parses the usage messages) in the code above, because it's quite long. In case anyone is interested, you can find it at https://code.google.com/p/xact-xtras/source/browse/interfacebuild.m

Here's an example of a function description in the source documentation notebook:

And here's how it looks once the above has been parsed and the package is loaded:

Answer 4

One possibility would be to define those formatted strings in their box form in the package files. It should be pretty straightforward to create boxes from your formatted expressions with ToBoxes and I think that result shouldn't suffer from the problems during copy and paste that you described. You could then use something like that for the definitions of your usage messages:

tst::usage = ToString[DisplayForm[
  RowBox[{"(",GridBox[{{StyleBox["m","TI"]},{StyleBox["n","TI"]}}],")"}]
], StandardForm];

where the middle line is what ToBoxes returns, the rest would be the same for every usage message. To some extent this is even editable directly in InputForm (depending on how familiar you are with the box structure), which could be done directly in the Workbench editor.

Answer 5

Assuming the use of a revision control system, which seems likely when you are after sophisticated documentation, I by far like Pillsy's answer the best. This answer is more or less a long comment to that answer.

What I don't like about Pillsy's solution is the Convert To > Standard Form Display step. This is because I and my collaborators are using the associated .nb notebooks not only to store usage strings, but also to be the main documentation system which non-developers of the library look at, including examples etc. The Standard Form Display, IMHO, is ugly compared to a text cell, and doesn't handle things like line breaks nicely.

My solution is to remove the Convert To > Standard Form Display step and instead get usageForms to actually deal with the usage string being in a text cell:

possibleBoxToString[elem_] := If[
  StringQ[elem],
  elem,
  ToString[
   DisplayForm[elem] /. 
    StyleBox[boxes_, "Input", args___] :> 
     StyleBox[boxes, "Input", FontFamily -> "Courier", args],
   StandardForm]
  ];

processCell[contents_] := If[
  StringQ[contents],
  contents,
  StringJoin@Map[possibleBoxToString, First@contents, 1]
  ];

usageForms = Cases[
   Get@"test.nb",
   Cell[contents_, ___, CellTags -> tag_, ___] /; 
     StringMatchQ[
      tag, __ ~~ "::usage"] :> (StringReplace[tag, "::usage" -> ""] :>
       processCell[contents]),
   Infinity
   ];

The idea here is to note that if a text cell has formatting, then the cell is of the form TextData[{item1,item2,...}] where the items are either boxes or strings. We use ToString[DisplayForm@expr,StandardForm] to convert any boxes to strings, and then StringJoin the converted items. Note the FontFamily replacement; just a personal preference.

Example:

Here's a documentation notebook, where the given cell has the tag QuantumChannel::usage:

Here's what the usage cell looks like: