# How to improve the creation of tables of code and comments

While writing a response to a certain MSE question I made a function that tabulates code and comments. (See the definition below.)

Here is an example:

code = "
FoldList[(* reduction function *)
Plus,(* function to apply repeatedly *)
0,(* initial value *)
{1,2,3,3,100}(* arguments in repeated computations *)]";
code,
"GridFunction" -> (Panel@Grid[#, Alignment -> Left] &)]

I have several problems with the implementation of GridOfCodeAndComments, the main one being that I have to give a string to the function instead of (commented) code.

For example, I would like to be able to write the tabulate code directly to GridOfCodeAndComments:

FoldList[(* reduction function *)
Plus,(* function to apply repeatedly *)
0,(* initial value *)
{1, 2, 3, 3, 100}(* arguments in repeated computations *)],
"GridFunction" -> (Panel@Grid[#, Alignment -> Left] &)]

How can this be done? Any suggestions would be appreciated.

Another, minor problem in GridOfCodeAndComments is that the pattern for matching comments, comPat, is somewhat weak. How can it be improved?

## Definition

Options[GridOfCodeAndComments] = {"GridFunction" -> (Grid[#, Alignment -> Left] &)};

Block[{grData, codeLines, commentLines, comPat, gridFunc},

gridFunc = OptionValue["GridFunction"];
If[TrueQ[gridFunc === Automatic], gridFunc = (Grid[#, Alignment -> Left] &)];

(* Split the code into lines *)
codeLines = StringSplit[code, "\n"];

(* Split each line into a {code, comment} pair *)
comPat = ("(*" ~~ (Except["*"] ..) ~~ "*)");
grData =
Map[
If[StringFreeQ[#, "(*"], {#, ""},
StringCases[#, (x__ ~~ y : (comPat) ~~ z___) :> {x <> z, y}][[1]]
] &, codeLines];

(* Style the code and comments *)
grData[[All, 1]] = Map[Style[#, "Input"] &, grData[[All, 1]]];
grData[[All, 2]] =
Map[Style[#, "CommentStyle" /. Options[$FrontEnd, AutoStyleOptions][[1, 2]]] &, grData[[All, 2]]]; (* Show result *) gridFunc[grData] ]; • By the time your function gets to the Kernel the comments have been stripped. What you could do, though, is use CellEvaluationFunction or$PreRead to prevent that. As for your comment matching, I think you might be better off splitting by "*)" than by line, then splitting at the first (*. Of course this assumes you don't have any (* or *) in your input code. That could be fixed by parsing on the box-structure, though and passing an alternate format to your function (as code, comment pairs, maybe). Otherwise it's not really worth it. – b3m2a1 Jun 12 '17 at 5:45
• related: 26136 – Kuba Jun 12 '17 at 8:16
• @b3m2a1 and Kuba thank you for your helpful comments! – Anton Antonov Jun 12 '17 at 11:45

Here's another possibility. Since your problem is fundamentally a problem of the comments being stripped, we can define an invisible wrapper Commented that evaluates away to nothing when operated on, and formats like a comment.

Here's a possible imp.

First make the formatting right:

Format[Commented[e_, c_]] :=
RawBoxes@
TemplateBox[
{
ToBoxes[Unevaluated@e],
ToBoxes[c]
},
"CommentedCode",
DisplayFunction ->
Function[
RowBox[{#, " ",
TemplateBox[{#2}, "Comment",
DisplayFunction ->
Function[

StyleBox[RowBox[{"(*", #, "*)"}],
ShowStringCharacters -> False]
]
]
}]],
InterpretationFunction ->
Function[RowBox[{"Commented", "[", #, ",", #2, "]"}]]
]

In[15]:= Commented[a, "test symbol"]

Out[15]= Commented[a, "test symbol"]

But if we look at its format form (i.e. copy it with Shift-Control-C):

a (*test symbol*)

Then make it invisible to evaluation:

Commented /: (h : Except[Hold | HoldForm])[a___, Commented[expr_, _],
b___] := h[a, expr, b];

1 + Commented[a, "test symbol"]

1 + a

Then you can define a function that will find all Commented annotations, like so:

Cases[HoldComplete[e],
Verbatim[Commented][a_,
b_] :> (HoldComplete[a] -> b), \[Infinity]];

And here's a chunk of nice formatted code to work from:

chunk = HoldForm@
Commented[
Table[
a~Commented~"Return the int",
{a, 1, 10}
],
"Create a list of ints"
]

Table[a (*Return the int*),{a,1,10}] (*Create a list of ints*)

Then:

{HoldComplete[a] -> "Return the int",
HoldComplete[Table[Commented[a, "Return the int"], {a, 1, 10}]] ->
"Create a list of ints"}

You can start to work with (an adapted form of) this data structure now, potentially more easily than before

• (+1) Interesting idea. One can also add Tooltip to formatted form of every Commented expression what may be very useful for teaching purposes. May be sometimes one will create a course Notebook bases on this idea, who knows. – Alexey Popkov Nov 12 '17 at 10:00
• Yes, very good and interesting idea. I will try to make the suggested "adapted form" and accept the answer soon. (Somehow, I did not see this answer until today...) – Anton Antonov Jul 21 '19 at 10:50

This answer is for a less general question:

• How to improve the creation of tables of code and comments for monadic pipelines?

As I mentioned in the formulation of the original question post, I am interested in making tables of code and comments in order to explain monadic programming. So, it occurred to me at some point that a special monad can be used to make those tables for monadic pipelines.

(To be clear, the problem gets simplified if we want to build code-comment grids for monad pipelines only.)

The resulting TraceMonad code is fairly simple, and demonstrates well the "programming semicolon" view of the binding operator in monadic programming.

I would say in this case the advice "eat your own dog food" is very useful -- it brings a nice solution (although a specialized one.)

In the example below note that :

1. the tracing is initiated by just using TraceMonadUnit;

2. pipeline functions (actual code) and comments are interleaved;

3. putting a comment string after a pipeline function is optional.

### Example

The example below has sparse explanations, but the TraceMonad file has fairly detailed ones.

Generate Maybe monad code for "Maybe":

Make up data:

data = {0.61, 0.48, 0.92, 0.90, 0.32, 0.11};

Execute a monadic pipeline and generate a table of code and comments:

MaybeFilter[# > 0.3 &] ⟹"(* filter current value *)"⟹
MaybeEcho ⟹"(* display current value *)"⟹
MaybeOption[(Maybe@Map[If[# < 0.4, None, #] &, #] &)]⟹"(* map values that are too small to None *)"⟹
MaybeEcho ⟹

• Hi @Anton Antonov, I tried the above code but the MaybeFilter stays blue and the output is not generated (the table of code is). At least I expect the filter to get rid of the items in the list > 0.3. Should I load another package? – Lou Nov 13 '18 at 13:31
• @Lou 1.) Sorry for my delayed reply! Somehow I missed this comment... 2.) "At least I expect the filter to get rid of the items in the list > 0.3." -- Your assumption is incorrect. MaybeFilter is the same as Select. In this case it keeps the items that are greater than 0.3 . This can be seen by comparing the echo output and the assignment to data. 3.) I will discuss the blue MaybeFilter in another comment. – Anton Antonov Nov 19 '18 at 15:24
• @Lou I forgot to include the code generation line GenerateMaybeMonadSpecialCode["Maybe"]. The code in my answer should work now, meaning MaybeFilter won't be blue. – Anton Antonov Nov 19 '18 at 22:38

This is a problem in design, and the chief difficulty in design is understanding the problem. Suppose you were charged with automating an algorithm in some complex subject area, say finite automata, so your organization could have fairly low-level workers give a it set of inputs and return a nicely formatted correct answer. So, your first action should be to write a set of requirements, preferably with a small set of input data, and a picture or diagram of what the output should look like. Define success.

Next, suppose you are given the finite automata algorithm to use or you find one in a text. You should also find a worked example, a problem at the end of the chapter, or make up a simple set of input data. The algorithm will be a few lines of English text, a few lines of (Boolean) algebra, a few lines of text, etc., etc., ....

For your problem, you should devise a written algorithm or recipe in a few lines of English text to go from simple input data to the desired output.

Now, on one of more sheets of paper, go thru the algorithm one line or sentence at a time to find an answer to your test problem. Written language is VERY imprecise, and it may hours or days before you really understand the algorithm author's (or your own) intent. Do this for several test problems until you really, really understand what the author means or what your problems really are. This can take a long time.

Next, translate your understanding of the algorithm to computer code. In Mathematica, you probably want to use the Module construct (Module[{constants, variables}, instructions, answer]), but you can also put all the initialization steps, constants, variables, instructions, answer in a single cell and just evaluate and re-evaluate that cell.

In these situations, I almost always use the Catch/Throw construct to incrementally arrive at the correct answer:

Catch[Module[{constants, variables},
one or more lines of code
print intermediate result
Throw[intermediate result]
] (* End Module *)

Compare the intermediate result with your worked example.

Repeat
Write a few lines of code
Compare result with example
until result is answer to problem

There is nothing worse than computer code the yields the wrong answer. The literature is full of horror stories of people who solved complex problems with Excel, only to find that when the algorithm was used in production, it cost the company thousands, and occasionally millions, of dollars to make it right with the customer. Hence, in Computer Science, Directive 0 is, "If it is not tested, it does not work."

Derive one or more new sets of data to test the solution to the problem. What happens when the inputs become really big, really small, and some big and some small?