Using DocumentationBuild and Transmogrify to generate web docs

Question

I build my docs in Mathematica, rather than using Workbench and the DocumentationTools. But now I want to be able to build HTML versions of my docs.

I knew Workbench (partially) supported this and so got the Workbench plugin for Eclipse, copied the DocumentationBuild` and Transmogrify` packages, and have now spent many hours trying to get them to work for me.

It seems to me that the packages are configured to work on the documentation templates that Workbench produces, but I want to adapt the Transmogrify process to work on a proper documentation notebook, meaning a documentation notebook as one would see them in the documentation center. E.g. Print (deep-linked to open in Mathematica)

I know this is not oft-used and wholly undocumented functionality, but is there any way I can do this?

(It seems like I may have to write a whole new set of XMLTransforms, but looking through them I don't quite know why they don't work and any guidance would be appreciated).

Answer 1

tl;dr

The Workbench packages are out-of-date for 11.1 and the whole system is not configured to easily work from Mathematica using a structure differing from the standard /language/ref URL structure or things like that.

It's a massive pain to make it all work, but in the end we can get half-way decent, largely-functional documentation pages like this

---

Shameless Self-Promotion:

I added this system + auto-generation into my DocGen package letting me build out docs like this PacletManager documentation using just the following:

(* Build out the docs from the contexts*)
GenerateDocumentation[
 {
  "PacletTools`",
  "PacletManager`",
  "DataPaclets`"
  },
 "Install" -> True
 ]

(* Generate HTML and web deploy from paclets *)
GenerateHTMLDocumentation[
 PacletFind[{"*Paclet*", "0"}],
 CloudDeploy -> True
 ]

---

Full Answer

To automate the entire process takes many tedious steps, but if you don't mind doing things by hand and follow Frank Martin's guide here and don't mind doing things by hand you only really need the first two steps.

Copy the packages from Eclipse

Install Workbench on Eclipse, as described here.

Then find the directory where the documentation packages are stored:

$eclipsePath = "path/to/Eclipse";
FileNames["DocumentationBuild", $eclipsePath, \[Infinity]]

Copy "DocumentationTools", "DocumentationBuild" and "Transmogrify" to FileNameJoin@{$UserBaseDirectory, "Applications"}, then delete the MenuSetup that's in "DocumentationTools" (just for safety)

11.1 to 11.0 Down Conversion

The heart of my initial problem was that the build tools I copied over from Workbench were out of date, so I got around that by down-converting my notebooks to 11.0 style before passing them through ExportWebPage.

Here's the code I used for that:

webExportNotebookPrep[nb_] :=
 DeleteCases[
   Visible -> False
   ]@
  ReplaceRepeated[
   nb, {
    Cell[TextData[{c_, ___}], "ObjectNameGrid", ___] :>
     c,
    Cell[_, "NotesThumbnails" | "ObjectNameTranslation", ___] :>
     Sequence @@ {},
    Cell[TextData[{Cell[BoxData[_DynamicBox], ___], e__}],
      t : "NotesSection" | "PrimaryExamplesSection" |
        "ExampleSection" | "ExampleSubsection", r___] :>
     Cell[TextData[e], t, r],
    DynamicBox[e : If[$VersionNumber < _, __], ___] :>
     e,
    Cell[TextData[s_String], r___] :> Cell[s, r],
    t : TemplateBox[{_, _String}, ___] :>
     refLinkRewind[t],
    TagBox[a : ActionMenuBox[_FrameBox, ___], 
      MouseAppearanceTag["LinkHand"]] :>
     a,
    Cell[
      TextData[{t_, " ",
        Cell[
         BoxData[
          GraphicsBox[{{
             {
              _GrayLevel, _Thickness,
              LineBox[{{-1.8, 0.5}, {0, 0}, {1.8, 0.5}}]
              }
             }},
           ___
           ]
          ],
         ___
         ]
        }],
      ___
      ] :>
     InterpretationBox[
      Cell[
       TextData[{
         t,
         StyleBox[" \[FilledDownTriangle]", "AnchorBarArrow",
          StripOnInput -> False]
         }]],
      TextCell[
       Row[{"See Also", 
         Style[" \[FilledDownTriangle]", "AnchorBarArrow"]}]
       ]
      ]
    }
   ]

With those corrections we can get out raw HTML, but there are still many issue to correct, which are largely documented by Frank Martin, which I dealt with programmatically.

---

File Post-Processing

Some resources we want to link do built for a specific directory layout, corrupted or inadequate.

We'll write a general purpose function for handling this post-processing.

webExportPostProcessSingleReplacements // Clear;
webExportPostProcessMultiReplacements // Clear;

(* MultiReplacements *)

webExportPostProcessMultiReplacements[strip_, res_, url_] :=
  With[{
    trimmedRes =
     StringTrim[res, "/" ~~ EndOfString],
    paddedRes =
     StringTrim[res, "/" ~~ EndOfString] <> "/"
    },
   Flatten@{(*
     (* Insert prototype into loaded JS characters *)

     "<script src='javascript/common.js' type='text/javascript'></script>\n"<>

          "<script src='javascript/jquery.js' \
type='text/javascript'></script>\n"->

     "<script src='javascript/jquery.js' type='text/javascript'></script>\n"<>

          "<script src='javascript/common.js' \
type='text/javascript'></script>\n",*)
     (* Relink loaded JS and CSS *)

       h : ("src" | "href") ~~ "=" ~~ (q : "'" | "\"") ~~ 
       t : "javascript" | "css" ~~ "/" :>

      h <> "=" <> q <> URLBuild@{res, t} <> "/",
     h : ("src" | "href") ~~ "=" ~~ (q : "'" | "\"") ~~ "/" ~~ 
       t : "javascript" | "css" ~~ "/" :>

      h <> "=" <> q <> URLBuild@{res, t} <> "/",
     "/webMathematica/Resources/Documentation/English/" ->
      paddedRes,
     (* Relink loaded JS and CSS *)

     l : ("<link rel=\"stylesheet\" href=\"" ~~
         (paddedRes | "") ~~ 
         "css/reference.css\" />") :>

      l <> "\n<link rel=\"stylesheet\" href=\"" <>

       URLBuild@{trimmedRes, "css", "clipboard.css"} <> "\" />",
     l : ("<script src=\"" ~~ (paddedRes | "") ~~ 
         "javascript/reference.js\"></script>") :>

      "<script src=\"" <>
       URLBuild@{trimmedRes, "javascript",
         "jquery", "core", "1.7.2", "jquery.min.js"} <> "\" /></script>\n" <> l
     }
   ];

(* SingleReplacements *)

webExportPostProcessSingleReplacements[strip_, res_, url_] :=
  With[{
    trimmedRes =
     StringTrim[res, "/" ~~ EndOfString],
    paddedRes =
     StringTrim[res, "/" ~~ EndOfString] <> "/"
    },
   Flatten@{
     Replace[strip, Except[_List | _Rule] -> {}],
     (* Protect true links *)

     "http://reference.wolfram.com/language/" ~~

       t : "ref" | "guide" | "tutorial" :>

      "http://reference.wolfram.com/language/" <> t,
     paddedRes ~~
       t : "ref" | "guide" | "tutorial" :>
      "http://reference.wolfram.com/language/" <> t,
     "/language/" ~~
       t : "ref" | "guide" | "tutorial" :>

      URLBuild@{
        "http://reference.wolfram.com/language",
        t
        },
     (* Patch intra site links *)

     "/language/" ~~ param : Except[WhitespaceCharacter] ... ~~

       t : "ref" | "guide" | "tutorial" ~~

       doc : Except[WhitespaceCharacter | "\""] ... :>
      URLBuild@
       DeleteCases[{url, StringTrim[param, "/"], t, StringTrim[doc, "/"]}, ""],
     "http://reference.wolfram.com/language" ->
      url,
     (* Patch resource directory *)
     "/common/" ->
      paddedRes,
     (* Patch jquery version *)

     "jquery/core/1.6.1" ->
      "jquery/core/1.7.2",
     (* Patch mathematicaImages to work a resource base *)

     "\"../" ~~ i : "images/mathematicaImages" :>

      "\"" <> URLBuild@{trimmedRes, i},
     "\"" ~~ i : "images/mathematicaImages" :>

      "\"" <> URLBuild@{trimmedRes, i},
     "\"/mathematicaImages" ->

      "\"" <> URLBuild@{trimmedRes, "images", "mathematicaImages"},
     (* Patch includes to work a resource base *)

     "\"/includes" ->
      "\"" <> URLBuild@{trimmedRes, "includes"},
     (* Downconvert 2013 resources to the standard resoruce base *)

     "/2013/" ->
      paddedRes,
     (* Delete duplicates *)
     (l : 
         "<script src=\"" <>

          URLBuild@{trimmedRes, "javascript", "jquery.min.js"} <> 
          "\" /></script>\n") .. :>
      l,
     (l : "<link rel=\"stylesheet\" href=\"" <>

          URLBuild@{trimmedRes, "css", "clipboard.css"} <> "\" />\n") .. :>

          l
     }
   ];

(* Extension specific replacements *)

webExportPostProcessSingleReplacements["css", strip_, res_, url_] :=
  {
   (* Add clipboard styles into reference.css *)

   t : "/*##############################\n\tCOLORS" :>
    "/* clipboard styles */\n@import url('clipboard.css');\n" <> t
   };

webExportPostProcessSingleReplacements["js", strip_, res_, url_] :=
  {
   (* Clean up dangerous characters while protecting whitespace *)

   w : WhitespaceCharacter :> w,
   _?(Not@*PrintableASCIIQ) -> ""   
   };

webExportPostProcessMultiReplacements[_, _, _, _] :=
  {
   };
webExportPostProcessSingleReplacements[_, _, _, _] :=
  {
   };

(* Main *)

webExportPostProcess // Clear

Options[webExportPostProcess] = {
   "ResourceBase" -> Automatic,
   "URLBase" -> Automatic,
   "StripExtensions" -> Automatic
   };
webExportPostProcess[file_String?FileExistsQ, ops : OptionsPattern[]] :=

  With[{
    s = ReadString[file],
    ext = Replace[OptionValue["StripExtensions"], 
      Automatic -> {".en." -> "."}],
    res =
     Replace[OptionValue["ResourceBase"],
      Automatic :>
       URLBuild@
        ConstantArray["..",
         With[{split = FileNameSplit@DirectoryName@file},
          SelectFirst[Range[Length@split],
           DirectoryQ@FileNameJoin@
              Append[
               Drop[split, -#],
               "javascript"
               ] &,
           1
           ]
          ]
         ]
      ],
    url =
     Replace[OptionValue["URLBase"],
      Automatic :>
       URLBuild@
        ConstantArray["..",
         With[{split = FileNameSplit@DirectoryName@file},
          SelectFirst[Range[Length@split],
           DirectoryQ@FileNameJoin@
              Append[
               Drop[split, -#],
               "javascript"
               ] &,
           1
           ]
          ]
         ]
      ]
    },
   If[StringQ[s],
    DeleteFile[file];
    Export[
     StringReplace[file,
      Replace[OptionValue["StripExtensions"], Except[_List | _Rule] -> {}]
      ],
     FixedPoint[#, s, 6] &@
       StringReplace@Join[
         webExportPostProcessMultiReplacements[
          ext,
          res,
          url
          ],
         webExportPostProcessMultiReplacements[
          FileExtension[file],
          ext,
          res,
          url
          ]
         ] //
      StringReplace@Join[
        webExportPostProcessSingleReplacements[
         ext,
         res,
         url
         ],
        webExportPostProcessSingleReplacements[
         FileExtension[file],
         ext,
         res,
         url
         ]
        ]
     ,
     "Text"
     ],
    file
    ]
   ];
webExportPostProcess[___] := $Failed

---

Resource copying

Then we'll take jkuczm's resource list to copy from this answer and copy each file to the documentation directory with requisite post-processing:

$webExportAssets :=
  Map[
   <|
     "Drop" -> FileNameDepth[#[[1]]] - FileNameDepth@#[[2]],
     "Files" -> FileNames[#[[3]], #[[1]], \[Infinity]],
     "Insert" -> If[Length@# > 3, #[[4]], Nothing]
     |> &,
   {
    {
     FileNameJoin@{
       DocumentationBuild`Common`$DocumentationBuildDirectory,
       "Internal", "web", "html", "javascript"
       },
     "javascript",
     {
      "*.js"
      }
     },
    {
     FileNameJoin@{
       DocumentationBuild`Common`$DocumentationBuildDirectory,
       "Internal", "web", "html", "css"
       },
     "css",
     {
      "*.css"
      }
     },
    {
     FileNameJoin@{
       DocumentationBuild`Common`$DocumentationBuildDirectory,
       "Internal", "web", "html", "2014", "standard"
       },
     "",
     {
      "*.js", "*.json", "*.css", "*.png", "*.gif", "*.jpg",
      "*.svg", "*.eot", "*.otf", "*.ttf", "*.woff", "*.html"
      }
     },
    {
     FileNameJoin@{
       DocumentationBuild`Common`$DocumentationBuildDirectory,
       "Internal", "web", "html", "2014", "standard"
       },
     "",
     {
      "*.ttf", "*.woff"
      },
     "fonts"
     },
    {
     FileNameJoin@{
       DocumentationBuild`Common`$DocumentationBuildDirectory,
       "Internal", "web", "html", "2014",
       "minimal",
       "javascript"
       },
     "javascript",
     "faster-page-load.js"
     },
    {
     FileNameJoin@{
       DocumentationBuild`Common`$DocumentationBuildDirectory,
       "Internal", "web", "html", "images",
       "mathematicaImages"
       },
     FileNameJoin@{"images", "mathematicaImages"},
     "bullet.gif"
     }
    }
   ];

webExportAssetsCopy[dir : (_String | _File)?DirectoryQ] :=

  Block[{d, f},
   Monitor[
    Table[
     With[{depth = a["Drop"], insert = a["Insert"]},
      Table[
       d =
        FileNameJoin@
         Flatten@{dir,
           DirectoryName@FileNameDrop[f, depth],
           insert
           };
       If[! DirectoryQ@d,
        CreateDirectory[d,
         CreateIntermediateDirectories -> True
         ]
        ];
       If[StringMatchQ[FileExtension[#], "css" | "js" | "html"],
          webExportPostProcess[#],
          #
          ] &@
        CopyFile[f,
         FileNameJoin@
          Flatten@{
            d,
            FileNameTake@f
            },
         OverwriteTarget -> True
         ],
       {f, a["Files"]}
       ]
      ],
     {a, $webExportAssets}
     ],
    Internal`LoadingPanel[
     "Copying `` to ``"~TemplateApply~
      {
       f,
       FileNameJoin@
        Flatten@{
          d,
          FileNameTake@f
          }
       }
     ]
    ]
   ];

---

HTML Generation

With all of that in place we can now generate pages that will look right:

webExportNotebook[file_, nb_, ops___?OptionQ] :=

 With[{params = Flatten@{ops}},
  Needs["DocumentationBuild`"];
  Block[{
    DocumentationBuild`Common`PubsEvaluateWithFE =
     Identity,
    DocumentationBuild`Common`Private`PubsStartFrontEnd =
     Identity,
    Print =
     Identity,
    NotebookOpen,
    NotebookPut,
    expFile =
     If[DirectoryQ@file,
      FileNameJoin@Flatten@{
          file,
          URLParse[
           Lookup[params, "URI",
            URLBuild@{
              "",
              "DocPage-" <> ToString[RandomInteger[10000]]
              }
            ],
           "Path"]
          } <> ".html",
      file
      ]
    },
   NotebookOpen[e___] :=
    MathLink`CallFrontEnd[
     FrontEnd`NotebookOpenReturnObject[e, Visible -> False]
     ];
   NotebookPut[e_] :=
    MathLink`CallFrontEnd[
     FrontEnd`NotebookPutReturnObject[Append[e, Visible -> False]]
     ];
   Quiet[
    DocumentationBuild`Export`ExportWebPage[
     ExpandFileName[expFile],
     webExportNotebookPrep@nb,
     Normal@params,
     Sequence @@
      FilterRules[
       Flatten@{
         ops,
         "CompleteHTMLQ" -> True
         },
       FilterRules[
        Options[DocumentationBuild`Export`ExportWebPage],
        Except["HistoryData" | "Language"]
        ]
       ]
     ]
    ]
   ]
  ]

Where we'll feed the result of gathering the notebook metadata parameters in as ops:

$webExportParameters =
  {
   "title" -> "Title",
   "titlemodifier" -> "TitleModifier",
   "history" -> "HistoryData",
   "type" -> "EntityType",
   "paclet" -> "PacletName",
   "context" -> "Context",
   "uri" -> "URI",
   "keywords" -> "Keywords",
   "tutorialcollectionlinks" -> "TutorialCollectionLinks",
   "summary" -> "Summary",
   "index" -> "IndexQ"
   };

webExportGatherParameters[nb_, ops___?OptionQ] :=
  Normal@
   KeyMap[
    Capitalize@*
     Replace[$webExportParameters],
    Association@
     FilterRules[
      Flatten@{
        ops,
        Fold[Lookup, 
         Options[nb, TaggingRules], {TaggingRules, "Metadata"}]
        },
      Alternatives @@ 
       Flatten[Through@{Keys, Values}[$webExportParameters]]
      ]
    ];

Note that the export process also blocks / changes a lot of things that Workbench uses that we don't need, e.g opens the notebook in hidden form, prevents a new front-end from being configured, and has the Transmogrify process work in quieted mode.

---

Composite function

We can then build all this into a single function:

Options[GenerateHTMLDocumentation] =
  Join[
   Options[GenerateSymbolPages],
   Options[webExportPostProcess],
   Options[DocumentationBuild`Export`ExportWebPage],
   Options[webExportAssetsDeploy], {
    CloudConnect -> Automatic,
    CloudDeploy -> False,
    "DeployAssets" -> False,
    "CopyAssets" -> Automatic
    }];
GenerateHTMLDocumentation[dir_String?DirectoryQ,
   nb : {__Notebook} | None,
   ops : OptionsPattern[]
   ] :=
  With[{
    deploy = OptionValue[CloudDeploy],
    highlight =
     CurrentValue[$FrontEndSession,
      {AutoStyleOptions, "HighlightUndefinedSymbols"}
      ]
    },
   If[
    TrueQ@OptionValue["CopyAssets"] ||
     And[
      OptionValue["CopyAssets"] === Automatic || 
       TrueQ@OptionValue["DeployAssets"],
      AnyTrue[{"css", "fonts", "images", "includes", "javascript"},
       Not@DirectoryQ[FileNameJoin@{dir, #}] &
       ]
      ],
    webExportAssetsCopy[dir]
    ];
   CheckAbort[
    Function[
      CurrentValue[$FrontEndSession,
        {AutoStyleOptions, "HighlightUndefinedSymbols"}
        ] =
       highlight;
      If[deploy,
       Replace[
        Replace[OptionValue[CloudConnect],
         Automatic -> Key["PacletsAccount"]
         ], {
         s_String :>
          If[$WolframID =!= s,
           CloudConnect[s]
           ],
         {s__String} :>
          CloudConnect[s],
         k_Key :>
          KeyChainConnect[k]
         }]
       ];
      If[deploy && OptionValue["DeployAssets"],
       Replace[
        Replace[OptionValue["ResourceBase"], {
          Automatic -> "reference",
          CloudObject[c_, ___] :> c
          }],
        s_String :>
         webExportAssetsDeploy[dir, s,
          FilterRules[
           {
            ops
            },
           Options@webExportAssetsDeploy
           ]
          ]
        ]
       ];
      If[deploy,
       Replace[
        Replace[OptionValue["URLBase"], {
          Automatic -> "reference",
          CloudObject[c_, ___] :> c
          }],
        s_String :>
         If[# =!= None,
          webExportCloudDeploy[dir, #, s,
           FilterRules[
            {
             ops
             },
            Options@webExportCloudDeploy
            ]
           ]
          ]
        ],
       #
       ]
      ]@
     If[nb === None,
      None,
      If[Length@nb > 1,
         Monitor[ReleaseHold[#],
          Internal`LoadingPanel[
           "Generating HTML page `` of ``"
            ~TemplateApply~
            {i, Length@nb}
           ]
          ],
         ReleaseHold@#
         ] &@
       Hold@
        Table[
         With[{
           params = webExportGatherParameters[nb[[i]], ops]
           },
          webExportPostProcess[
           webExportNotebook[dir, nb[[i]], params],
           FilterRules[
            Flatten@{
              If[deploy,

               "URLBase" ->

                Replace[OptionValue["URLBase"], {

                  Automatic :>

                   First@CloudObject["reference"],
                  CloudObject[u_, ___] :>
                   u,
                  s_String :>
                   First@CloudObject[s]
                  }],
               Nothing
               ],
              If[deploy,

               "ResourceBase" ->

                Replace[OptionValue["ResourceBase"], {

                  Automatic :>

                   First@CloudObject["reference"],
                  CloudObject[u_, ___] :>
                   u,
                  s_String :>
                   First@CloudObject[s]
                  }],
               Nothing
               ],
              params
              },
            Options[webExportPostProcess]
            ]
           ]
          ],
         {i, Length@nb}
         ]

      ],
    CurrentValue[$FrontEndSession,
      {AutoStyleOptions, "HighlightUndefinedSymbols"}
      ] =
     highlight;
    $Aborted
    ]
   ];

Then we merge it with the automatic documentation generation process I wrote up here:

GenerateHTMLDocumentation[
   dir : (_String | _File)?
      DirectoryQ | _FileName?(DirectoryQ@*ToFileName),
   pattern : _String?(Not@*FileExistsQ) | _Symbol,
   ops___?OptionQ
   ] :=
  With[{n = Names[pattern]},
   If[Length@n > 1,
    $Failed,
    With[{
      nb =
       First[{NotebookGet[#], NotebookClose[#]}] &@
        First@Flatten@List@
           GenerateSymbolPages[
            Evaluate@First@n,
            Evaluate@FilterRules[{ops}, Options@GenerateSymbolPages],
            Visible -> False
            ]
      },
     GenerateHTMLDocumentation[
      Replace[dir, f_FileName :> ToFileName[f]],
      nb,
      ops
      ]
     ]
    ]
   ];

And then let it generate the docs without specifying a directory:

$webDocsHTMLBuildDirectory =
  FileNameJoin@{$TemporaryDirectory, "web_docs"};
GenerateHTMLDocumentation[
  Optional[Automatic, Automatic],
  s___
  ] :=
 With[{dir =
    Quiet[
     DeleteDirectory[$webDocsHTMLBuildDirectory, 
      DeleteContents -> True];
     CreateDirectory[$webDocsHTMLBuildDirectory]
     ]},
  GenerateHTMLDocumentation[
   dir,
   s
   ]
  ]

---

Viewing / Deploy the Docs

Some things in the docs require HTTP, so to view the docs we make we'll want a function to start a server in a directory. It doesn't look like HTTPHandling`StartWebServer does that so we'll need something else. I chose to use python's SimpleHTTPServer

I put some sugar on it in my PyTools package. Look for PySimpleServerStart and PySimpleServerOpen.

But long-term having local docs is rather useless so I chose to put mine in the cloud, using these two functions to copy the resources and files up:

Options[webExportAssetsDeploy] =
  DeleteDuplicatesBy[First]@
   Join[
    {
     Permissions -> "Public"
     },
    Options[CloudObject]
    ];
webExportAssetsDeploy[
  dir_String?DirectoryQ,
  uri_String,
  ops : OptionsPattern[]
  ] :=
 With[{
   files =
    Select[
     FileNames[
      "css" | "fonts" | "images" | "includes" | 
        "javascript" ~~
       $PathnameSeparator <> "*",
      dir,
      \[Infinity]],
     Not@*DirectoryQ
     ]
   },
  Block[{f},
   Monitor[
    Table[
     CopyFile[
      f,
      CloudObject[
       URLBuild@
        Flatten@{uri, 
          FileNameSplit@FileNameDrop[f, FileNameDepth@dir]},
       Sequence @@ FilterRules[Flatten@{
           ops,
           Options[webExportAssetsDeploy]
           },
         Options[CloudObject]
         ]
       ]
      ],
     {f, files}
     ],
    Internal`LoadingPanel[
     "Copying `` to ``"~TemplateApply~
      {
       f,
       URLBuild@Flatten@{uri,
          FileNameSplit@FileNameDrop[f, FileNameDepth@dir]}
       }
     ]
    ]
   ]
  ]

Options[webExportCloudDeploy] =
  DeleteDuplicatesBy[First]@
   Join[
    {
     Permissions -> "Public"
     },
    Options[CloudObject]
    ];
webExportCloudDeploy[
  dir_String?DirectoryQ,
  pages : {__String?FileExistsQ},
  uri_String,
  ops : OptionsPattern[]
  ] :=
 With[{
   files =
    Join[
     pages,
     Select[
      FileNames[
       "Files" <> $PathnameSeparator ~~

        Alternatives @@ 
         Map[FileBaseName, pages] ~~
        $PathnameSeparator <> "*",
       DirectoryName@First@pages,
       \[Infinity]],
      Not@*DirectoryQ
      ]
     ]
   },
  Block[{f},
   Monitor[
    Table[
     Take[#, 1] &@
      CopyFile[
       f,
       CloudObject[
        URLBuild@
         Flatten@{uri,
           FileNameSplit@FileNameDrop[f, FileNameDepth@dir]},
        Sequence @@ FilterRules[
          Flatten@{
            ops,
            Options[webExportCloudDeploy]
            },
          Options[CloudObject]
          ]
        ]
       ],
     {f, files}
     ],
    Internal`LoadingPanel[
     "Copying `` to ``"~TemplateApply~
      {
       f,
       URLBuild@Flatten@{uri,
          FileNameSplit@FileNameDrop[f, FileNameDepth@dir]}
       }
     ]
    ]
   ]
  ]

(Note that they're already included in GenerateHTMLDocumentation)

---

End Product

After all of that we have docs that are (mostly) functional and in the cloud. Here's an example of what this is good for:

In[60]:= GenerateHTMLDocumentation[ 
 GenerateHTMLDocumentation,
 CloudDeploy -> True, 
 "CopyAssets" -> False
 ]

Out[60]= CloudObject["https://www.wolframcloud.com/objects/b3m2a1.docs/reference/BTools/ref/GenerateHTMLDocumentation.html"]

If we then go to https://www.wolframcloud.com/objects/b3m2a1.paclets/docs/BTools/ref/GenerateHTMLDocumentation.html we get a mostly functional 11.0 style documentation page:

Answer 2

Starting in Wolfram v13.0, there is initial official in-language support for HTML documentation builds, via the new PacletDocumentationBuild function. Also new in v13.0 is the Palettes > Documentation Tools palette, which is used to author new documentation pages.

Using Documentation Tools and PacletDocumentationBuild, it is now possible to author and build documentation entirely from within Mathematica / Wolfram Desktop.

However, there are some caveats. While these tools are available and useable right now, we're still working on fully documenting how to best use them. And the HTML building functionality is still new, and the rendered HTML may have visual glitches.

Some notes:

• The layout and styling of the web content produced by PacletDocumentationBuild is up-to-date — these are the latest styles used by the official Wolfram Language documentation at reference.wolfram.com.

  • PacletDocumentationBuild uses the same code as we do internally to build the Wolfram Language documentation notebooks and web docs. It will stay up to date as time goes on.

• PacletDocumentationBuild is specifically intended for building documentation that is part of a paclet. If you have documentation that is not part of a paclet, you will need to create a paclet for it in order to use this function.

• You can build and publish HTML directly to the cloud:

  PacletDocumentationBuild["/path/to/YourPaclet", CloudObject["MyPacletDocs"], "HTML"]
  

See my answer to "What are the best resources for paclet development?" to learn more about creating paclets and authoring documentation using these tools.