21
$\begingroup$

I know many people here have various websites associated with them and I'm guessing do at least some of the development of that stuff in Mathematica.

I recently fell down that rabbit hole myself, building out a website with Mathematica by using pelican as the generator (but running pelican from within Mathematica). And I was wondering what other Mathematica-base ways there are to build out such sites.

I'm fond of the customizability of my route, but with great customizability comes great potential to majorly screw things up. Also I'm using the cloud which is known to be buggy, so it's only a matter of time before my site goes up in smoke.

I know C.E. has a WordPress plugin. Is that the standard route?


Update:

One other method I forgot about is that Transmogrify was built for this purpose. But I've never used it for anything other than documentation building as it seems to be a bit finnicky and takes a lot of work to use. Has else anyone used it for anything other than docs?


I guess my question is, what are good route for building full websites with Mathematica? They don't have to be hosted in the cloud, but to restrict the scope of this question, I'm interested in methods that use Mathematica in some integral way--i.e. to deploy the site or, at the minimum, to generate the markdown--not just to make figures.

If it's still too open ended / subject to opinion I'll see if I can restrict the scope further.

$\endgroup$
  • $\begingroup$ "[...] building out a website with Mathematica by using pelican as the generator [...]" -- impressive undertaking. $\endgroup$ – Anton Antonov Jul 30 '17 at 21:00
  • $\begingroup$ @AntonAntonov it was actually only ~1 days work for the actual building code because pelican is so easy to use (and StartProcess is too). Theme development actually took longer, because I'm not a web programmer. $\endgroup$ – b3m2a1 Jul 30 '17 at 21:03
  • $\begingroup$ @Nasser using Markdown it's not so bad, really, if you know your box forms. I never thought about using LaTeX for it (as I never bothered to learn to use LaTeX). What do you mean you write LaTeX to file as it runs, but not exporting the NB? You have selected cells styles that you save to TeX, instead of the entire NB? $\endgroup$ – b3m2a1 Jul 30 '17 at 21:31
  • $\begingroup$ @Nasser I'm afraid I don't do much "math-heavy" content, myself. I'm much more on the chemistry side of things. Mind posting your LaTeX generating method as an answer? Proper math people may find that useful. $\endgroup$ – b3m2a1 Jul 30 '17 at 21:36
  • $\begingroup$ @Nasser also how do you manager your hosting? $\endgroup$ – b3m2a1 Jul 30 '17 at 21:37
13
$\begingroup$

In my opinion, Mathematica's XML template feature makes this quite easy. You'd write your HTML code for each page using XML templates, and then populate the templates with data as necessary using TemplateApply. The collection of XML templates that make up the website would be what other systems call a "theme". This can be used either to build a static website offline and then deploy it, or it can be uploaded to the cloud where it might be used to dynamically generate a website based on information that the backend code pulls from databases, etc.

To make the website maintainable, recurring parts such as the menu may be put in its own XML template and then requested as needed in other parts of the website. Thus, to change the menu one would only have to change it in one place. Splitting a design into modules like this is the main idea behind most templating systems, and the fact that Mathematica's XML templates are capable of it means that they can go a long way.

WRI is pushing for XML templates to be considered for this type of website building scenarios and probably developed it with website building in mind, although they will typically say report generation rather than "website generation". In practice, report generation is not different from generating a website.

EDIT: b3m2a1 has now implemented a package that builds websites following this approach. It is available here.

$\endgroup$
  • $\begingroup$ That is a very neat idea. Do you know if WRI is going to release a better NB -> XML transform system? The two of those together would make a system like pelican very simple to implement. Add some base theme to work off of and it'd be great. $\endgroup$ – b3m2a1 Jul 30 '17 at 23:26
  • $\begingroup$ @b3m2a1 Why is NB -> XML needed for this? $\endgroup$ – C. E. Jul 30 '17 at 23:28
  • $\begingroup$ My assumption was that you'd write a notebook to be passed into TemplateApply (or some derived function) to generate a page. That's the most natural way to approach things from my perspective. The NB -> XML would happen on the backend I guess. $\endgroup$ – b3m2a1 Jul 30 '17 at 23:29
  • $\begingroup$ @b3m2a1 You mean using a notebook as a WYSIWYG editor? Well, I think you have shown, and I too in my Mathematica-to-WordPress code, that it is quite possible to convert those cell expressions that matter... $\endgroup$ – C. E. Jul 30 '17 at 23:48
  • 1
    $\begingroup$ Done now (plus I made it a little easier to build a site). I'll put in a note about how it's basically a reimplementation of pelican once some documentation stops building in an hour or so. $\endgroup$ – b3m2a1 Aug 27 '17 at 17:19
12
$\begingroup$

This small note describes the method I use. I've been using this method for years. (all of my web site is written in Latex actually, and converted to HTML. First I used to use latex2html then switched to tex4ht few years ago as it is better supported and it comes with texlive).

To use Mathematica to generate HTML, the idea is to mix Latex inside the notebook, and generate the computation result to latex file on the fly as the program runs.

enter image description here

This method have many advantages, but it requires one to know some Latex to use it.

It uses Latex as the gateway between CAS and HTML. CAS generates Latex, tex4ht converts Latex to HTML. pdflatex converts Latex to PDF.

There are mainly two method to mix CAS generated computation with Latex.

enter image description here

The first method is what packages such as pythontex uses. The second method is what I think is the better method. It is more robust.

Advantages of generating Latex from Mathematica (instead of say HTML) are many:

First, one gets a PDF file for free along the way. This is huge advantage.

Second, one gets all the features that comes with Latex/tex4ht, such as automatic table of contents generation (with HTML links to other pages), automatic href link generation from inside the document, breaking large document into many smaller HTML pages and having all the HTML links automatically generated, automatic bibliography generated to HTML, And many more. (try to do all this with manual HTML coding :)

So Mathematica is used only to generate Latex. TeXForm is used heavily in this process. The rest is done by tex4ht and pdflatex if one wants pdf file as well.

I use this method all the time. As an example, I just generated this 300 pages document by a program I wrote in Mathematica over one week time, which Shows the step by step solution of simple 1st order ODE's from textbooks. Converted to HTML pages with tex4ht.

To illustrate this method, I used the example posted here.

To do this easily, CODE input cell is better suited for this than INPUT cell (since lots of Latex code is used inside the notebook, it is easier to format using CODE cell)

I'll post the code used. Here is a link to HTML page generated and the PDF file as well and the notebook I used. (will add small screen shot below as well)

Here is the code listing as well (same as in the above link) (this can be improved more, it was done quickly. This actually builds everything into one string. But in actual reports, I send Latex output to file immediately).

SetDirectory[NotebookDirectory[]];
ClearAll[x,s];
tex[s_] := ToString@TeXForm[s];

fileName ="index.tex";
file=OpenWrite[fileName,PageWidth->Infinity];

from=0; to=1;
int=HoldForm[Integrate[x^2,{x,0,1}]];

s="
\\documentclass[11pt]{article}
\\usepackage{amsmath} 
\\usepackage{graphicx}
\\usepackage{array}
\\usepackage[letterpaper,  margin=1in]{geometry}
\\usepackage{longtable}
\\usepackage{float}
\\usepackage{hyperref}
\\begin{document}

\\ifdefined\\HCode
\\href{index.pdf}{PDF file here}
\\href{example_making_web_page.nb}{Mathematica notebook used to generate the Latex}
\\fi

\\title{Web page generated using Mathematica to Latex to tex4ht to HTML process}
\\date{\\today}
\\author{By Mathematica, Latex and tex4ht}
\\maketitle
\\tableofcontents
\\section{First example}
   The integral of $x^2$ from $"<>tex[from]<>"$ to $"<>tex[to]<>"$ is 
   \\[
      \\boxed{ " <>tex[int]<>" = "<> tex[ReleaseHold[int]]<>"}
   \\]
   From $0$ to $2 \\dots 4$ the integral evaluates to 
   \\begin{itemize}
";

  Do[
     With[{n=i},
          int=HoldForm@Integrate[x^2,{x,0,n}];
          s=s<>"\\item $"<>tex[int]<>" = "<> tex[ReleaseHold@int] <>"$\n"
     ]
    , {i,2,4}
  ];  
  s=s<>"\\end{itemize}
  The values of $\\sin(x)$ in an inteval of $0$ to $3$ in $0.1$ steps (rounded to $10^{-4}$ are 
  \\["<>
  tex[TableForm@Transpose@Table[{x,N@Round[Sin[x],10^-4]},{x,0,3,1/3}]]<>
  "\\]
  The Plot for $\\cos(x)$ is";
  p=Plot[Cos[x], {x, 0, 2 Pi},Frame->True,PlotStyle->Red,GridLines->Automatic,GridLinesStyle->LightGray];
  Export["myImage.pdf",p];
  s=s<>"
     \\begin{figure}[H]
     \\centering
        \\includegraphics[width=0.5\\textwidth]{myImage}
     \\caption{my plot of $\\cos(x)$}
     \\end{figure}
  ";

s=s<>"
     \\section{Second example}\n";
     functions = {Sin, Cos, Tan, Sqrt, RiemannSiegelZ}
     s=s<>"\\begin{center}
           %\\begin{table}
           \\begin{longtable}{|l|m{.7\\textwidth}|}\\hline \n";
     Do[
         s=s<>"$"<>tex[functions[[i]]]<>"$&";
         p=Plot[functions[[i]][x], {x, 0, 3},Frame->True,ImagePadding->20,GridLines->Automatic,GridLinesStyle->LightGray];
         imageName = "myImage"<>ToString[i]<>".pdf";
         Export[imageName,p];
         imageName = "myImage"<>ToString[i];
         s=s<>"\\includegraphics[width=0.3\\textwidth]{"<>imageName<>"}\\\\ \\hline \n"
         , 
        {i, 1, Length@functions}
     ];
     s=s<>"\\end{longtable}
           %\\caption{My table of functions}
           %\\end{table}
           \\end{center}";
     s=s<>"\\end{document}";
     WriteString[file,s];
     Close[file];

Running the above generates index.tex in the same folder. This file is then compiled using the command make4ht index.tex which generated the HTML pages.

enter image description here

$\endgroup$
  • 1
    $\begingroup$ Seems like a good workflow, if a little bit user intensive (I'm more of the type an article, save it, and click some buttons type of person). I'm sure people will find this useful. And theoretically one could use the cloud as a free host for such pages. $\endgroup$ – b3m2a1 Aug 1 '17 at 1:27
  • $\begingroup$ @b3m2a1 as I said, this is for when is doing allot of computation and one wants to generate a Latex report and HTML pages from it. With this method, to make any changes, one simply re-run the notebook and new PDF and new web page is generated automatically. Change an equation? re-run the notebook and get new report. It is all automatic. If one has no computation to do, then this method is not needed ofcourse. $\endgroup$ – Nasser Aug 1 '17 at 1:29
  • $\begingroup$ @b3m2a1 I am a littel confused actually. If you just want to type some article and have that convert to HTML, then can I ask why you need Mathematica for? Would not just using Latex or markeup if you want be the most simple solution? Type the text, click Compile to HTML. Done. I use texstudio for this. Very good editor. Since there is no computation involved, I do not see why you need Mathematica. I assumed you want to generate a web pages content from some computation. $\endgroup$ – Nasser Aug 1 '17 at 4:16
  • $\begingroup$ See this as an example of what I'm dealing with. I tend to want to be able to work in a notebook and have almost exactly what I have in my notebook show up there. Markdown / LaTeX are kinda troublesome and not quite natural enough for what I generally like to do. Plus I usually have Mathematica running and do my explorations there anyway. $\endgroup$ – b3m2a1 Aug 1 '17 at 4:20
  • $\begingroup$ and have almost exactly what I have in my notebook show up there OK, I am still little confused. If all what you want is convert a notebook to HTML, then why not simply hide all the cells you do not want to show up, then just click on File->Save As->Web page ? Then upload the HTML page to your web page? But this is too obvious, so I must still be not understanding clearly what is it exactly you are trying to automate :) $\endgroup$ – Nasser Aug 1 '17 at 4:28
11
$\begingroup$

This is not an anwser to produce live/production code for websides. (yet)

I guess it's not what you want because its not very sophisticated but one could use MMA to make a good html code emitter directly from Mathematica Expressions.

We just write a function html which repeatedly calls itself on the subexpressions till it reaches "atomics" we can directly translate to html and afterward StringJoin'em.

I made a small proof of concept. Theoretically this could be made into a huge library with direct intrinsic functions and normal Mathematica expressions. (The code supports strings, unordered lists, Grids and graphics(with auto export))

html[params__] := 
 StringJoin["<html>\n<body>\n", html /@ List[params], 
  "\n</body>\n</html>\n"]

html[param_String] := param

html[param_List] := 
 StringJoin["\n<ul>\n", 
  StringJoin["<li>", #, "</li>\n"] & /@ (html /@ param), "</ul>\n"]

html[param_Grid] := Module[{data = param[[1]]},
  StringJoin["<table>\n", Table[
    StringJoin["<tr>\n", 
     Table[StringJoin["<td>", html[data[[rows, columns]]], 
       "</td>\n"], {columns, 1, Length[data[[rows]]]}], "</tr>\n"]
    , {rows, 1, Length[data]}], "</table>"]
  ]

html[param_Graphics] := 
 Module[{name = StringJoin[CreateUUID["plot-"], ".png"]}, 
  Export[name, param]; StringJoin["<img src=\"", name, "\"></img>"]]

html[param_] := ToString[param, InputForm]

This can be called with as many Mathematica expressions you want to. At example:

html[
 "The integral of x^2 from 0 to 1 is ",
 Integrate[x^2, {x, 0, 1}],
 "\nfrom 0 to 2, 3, or 4 the integral evaluates to",
 Table[Integrate[x^2, {x, 0, i}], {i, 2, 4}],
 "\n\nThe values from sin(x) in an inteval of 0 to 3 in 0.1 steps are \
(rounded to 10^-4)",
 Grid[Transpose[Table[{x, N@Round[Sin[x], 10^-4]}, {x, 0, 3, 1/3}]]],
 "\n\nThe Plot for cos(x) is\n\n",
 Plot[Cos[x], {x, 0, 2 Pi}]
 ]

generates the coresponding code to create this webside:

The generated webside

The generated code for this is:

generated code

Annother example

functions = {Sin, Cos, Tan, Sqrt, RiemannSiegelZ, LegendreP[2, #] &};
html[
 Grid@Table[{functions[[i]], 
    Plot[functions[[i]][x], {x, 0, 3}, ImageSize -> 300]}, {i, 1, 
    Length@functions}]
 ]

enter image description here

Also a css style generator could be written into this whole thing. So if you have enough spare time, you could build awesome things in this way. (I feel like, this would give a nice package)

$\endgroup$
  • 2
    $\begingroup$ Interesting idea. I've done some similar stuff myself, but never with much sophistication. If I can make one suggestion it's to use XMLElement. Much easier for recursive code formatting. My package for that lives here. But as I said, it never really got off the ground. $\endgroup$ – b3m2a1 Jul 30 '17 at 21:33
9
$\begingroup$

I thought I'd just briefly lay out what I do so people don't have to go to the link to my blog post in the question. The packages for all this junk are here, here, and here. Look for the things with Pelican and Py starting their names and the WebSiteDeploy function. The autodoc pages are here, here and here

Step 1: pelican

I wrote a venv wrapper, installed pelican using that via pip, setup up a Mathematica palette to handle making new sites and posts, made a theme, etc.

Step 2: Notebook to Markdown

Then I wrote a Notebook to Markdown converter because pelican can handle Markdown. Basically all complex box forms get rasterized. Math forms will be included once I finally get around to writing something about discrete variable representation and need them. Pre-rasterization future file names are checked against existing ones to avoid unnecessary rasterize calls.

Step 3: Site Deployment

Using that same venv wrapped I call pelican's build process, then deploy the new content using WebSiteDeploy

Step 4: Palettes / Stylesheets

I also built out a stylesheet that also saves to Markdown when a standard {"MenuCommand", "Save"} is called and a palette for the build processes, new file making, stuff finding, etc.

It's overkill and with a bit of work C.E.'s version is clearly a better way to do the same in the future. But it's good to have out here, I think.

$\endgroup$
3
$\begingroup$

Here's my second method, based on C.E.'s idea of using XMLTemplate.

I've detailed exactly what I did here

This is a site that I built with my generator

My process basically went as follows:

1. XMLTemplates

For your site theme, you need to define a collection of XMLTemplate files. You can see one of these here.

For new sites I just branch off of that

2. XMLTemplate function library

This allowed me to greatly simplify the quantity of code I had to right. My lib of functions can be found here.

The framework that creates the site exposes these by default

3. Write extractor for XMLObject

This involved pulling both metadata and content from a the XMLElements. That basically just required a few Cases calls

3a. Write Markdown to HMTL converter

I already had notebook to markdown, so then I had to write markdown to XMLObject. This is incomplete. If you have a more complete implementation, please add it as an answer to this question

4. Create site-data cache

I defined a standard symbol that caches all of the data for a given site. This is then used in the XMLTemplates.

5. Generate all site content

This just outputs to a standard directory, like pelican does

6. Deploy

Same as what I did with pelican

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.