21

From my understanding of what's going on, Mathematica does the following when you hit "Run": Parse the entire test notebook to look for input test cells and get the corresponding cell ids. Run the tests and collect the outcomes (using the cell ids from step 1). Generate test stats (total tests run, successes, failures) and provide links to jump to the next ...


13

What R.M. answer means is that Get as a part of the test will not help. The reason is that the content is parsed before Get is evaluated so needed contexts are not present yet. Additionally the test is run inside Block[{$Context},...] so manual Get before hitting Run won't fix that either because Block leaves only System` and Global` visible during content ...


12

This has been partially answered before, so here I will highlight some of its evolution since that previous answer. VerificationTest and its MUnit` antecedents do not have a mechanism for introducing new error types, so we need a function to do that for us. For this, I use a function called checkGraphicsRendering (outlined at the end) with the general use ...


12

Testing notebooks or .wlt test files? I use both. I create the tests in a notebook, but keep a .wlt file for automated test runs. This is my workflow: I start by creating a new "Testing Notebook" from the file menu I add in my testing inputs. For each test, I put all inputs in one cell, and return a single expression. Select all the cells and use the "...


11

After I have spent some time developing my package MeshTools and experimenting with testing workflows I am coming back to report my experience. I have put tests (VerificationTest) in a .wl file and this currently satisfies my requirements. I can conveniently edit it in the FrontEnd (see figure below), code is saved in InputForm rather than FullForm and can ...


10

You can make it look somewhat more pleasant by using simple expression parsers. For example, define an auxiliary head Tests, as follows: ClearAll[transformTest]; SetAttributes[transformTest, {HoldAll, Listable}]; transformTest[lhs_ -> rhs_] := Hold[VerificationTest[lhs, rhs]]; transformTest[Tests[tests___]] := Thread[transformTest[{tests}], Hold]; ...


9

I think @Jason B.'s answer is a good workflow. Here is an alternative that I have adopted over the years. Most of my code I write in *.m / *.mt / *.wl / *.wlt files using IntelliJ Idea with the Mathematica Plugin, so version control is straightforward. My projects are written as paclets (see How to distribute Mathematica packages as paclets? for an ...


8

I am not aware of something ready to use for fully automated testing of Mathematica code, but the building blocks for that are available in newer Mathematica versions. The following is just a rough description of the steps necessary for such a setup. As I think it would be valuable to have a more detailed recipe for such a setup I made this a community wiki,...


7

This seems to be a bug in .../SystemFiles/Components/MUnit/Kernel/Notebooks.m, probably somewhere in testOptionPatterns or whiteSpaceOrNone or somewhere there. Of course it is nearly impossible to efficiently parse any longer NotebookRead box structure with the Kernel pattern matcher, so I would always run test files from .mt files in Workbench, or, if you ...


6

The most straightforward method seems to be to map the test property you are looking for over the TestReportObject, as follows: report = TestReport[{VerificationTest[Limit[Exp[t], t -> 0], 2], VerificationTest[PrimeQ[7]], VerificationTest[Factor[1 - 2 x + x^2], (x + 1)^2], VerificationTest[Infinity - Infinity, Indeterminate]}]; Map[#["...


6

From the comments, The third argument of Test is the list of expected messages, e.g. Test[1/0, ComplexInfinity, {Power::infy}] The Mathematica system function VerificationTest is based on Test and has a similar design.


5

The answer below is for the first version of the question: I'd really like to receive the messages from the integrand. How can I stop NIntegrate from suppressing the messages? There are several reasons for this behavior. Generally speaking, NIntegrate pre-evaluates and localizes the integrands in order to facilitate different symbolic pre-processing and ...


5

This was the first time I played with Test Notebook so maybe I missed something obvious. But it works for OP so: (Composition[ CellPrint, ReplacePart[#, 2 -> "Input"] &, NotebookRead ] /@ Cells[ # , CellStyle -> "VerificationTest"] )& One should apply that function to the Notebook Object of ...


5

The culprit in some plots is that an Annotation with an internal tag is wrapped around the graphics elements. In others (many *Chart plotters), the culprit is a DynamicModule variable that has a unique symbol. Below is a fix for Annotation, for which the undefined symbol Congruent, or ESC===ESC, seems an appropriate choice. (In practice, in code other ...


4

Wolfram Workbench uses MUnit` package to run tests. MUnit` provides TestRun function that can be used from Mathematica (Workbench doesn't use this function, it has special test runner written in Java, but that would be hard to use for what you want). To use TestRun function, first you need to load MUnit` package. If you're using Mathematica version 10, you ...


4

I finally found a workaround to the message suppression that still allows the subdivision of the interval of integration $[0,3]$ at $x = 1, 2$. You have to protect the message-generating code sufficiently deep so that the symbolic processing takes care of the singularities. Then inside a ?NumericQ protected function, turn on messages again by resetting $...


4

The use of FileNameJoin seems to trip up the unit tester (this is a bug). A workaround is to just use / as a path separator in the string (works on OSX, Linux, and Windows): TestSuite[ { "ExternalFunctions/GCS_routines/GCSroutines.mt" , "TestCases/SphericalHarmonics.mt" , "TestCases/IgesRead.mt" } ]


4

I think this question is a mistake from my side but I also think this answer may be useful for future visitors. testing involves low level notebook programming It seems I was just expecting from this to be too forgiving, if you make your functions self contained, independent from EvaluationNotebook[], it will work quite fine. Module[{nb, oCell}, nb = ...


4

The fix is to realize which TaggingRule testing notebook cares about and overwrite all with this one: nb = CreateNotebook["Testing"]; SetOptions[nb, TaggingRules -> {"$testsRun" -> False}]; Rest @ NotebookGet @ nb My final goal was to export a testing notebook without prompting a visible notebook. Unfortunately CreateNotebook does not accept options :...


4

Your equation might be transformed to 2 Cot[x] == x/p - p/x with a new parameter p= (h (L/n) )/k With p0=(h (L/n) )/k /. {L -> 0.25, n -> 20, k -> 16, h -> 0.1}; (*0.000078125*) you might visualize the solution with ContourPlot ContourPlot[ContourPlot[1/(2 Cot[x]) == 1/((x/p) - p/x), {x, 0, 60}, {p, 0, 2 p0},FrameLabel -> {x,"p=\!\(\*...


3

If you want to locally override behavior of some symbols you can use Block which will completely remove original behavior, or Internal`InheritedBlock which will keep original behavior that was not explicitly overridden. ClearAll[testF, kkk] testF[] := kkk[] kkk[] = 0; testF[] (* 0 *) Block[{kkk}, kkk[] = 1; testF[] ] (* 1 *) testF[] (* 0 *) We can ...


3

I think it's useful to know all possible forms of expected messages accepted by Test. It accepts MessageName expression and list of such expressions (as already shown in ilian's answer): Test[Message[f::argx, f, 2], Null, f::argx] Test[Message[f::argx, f, 2]; Message[g::argr, g, 3], Null, {f::argx, g::argr}] Test also accepts Message expression and list ...


3

You can make your own code work simply enough by controlling evaluating with e.g. Inactive Inactive[TestReport][ Inactive[VerificationTest] @@@ Thread[{Inactive[StringFreeQ] @@@ args, results}]] // Activate or Block: Block[{StringFreeQ, VerificationTest, TestReport}, TestReport[VerificationTest @@@ Thread[{StringFreeQ @@@ args, results}]] ] If ...


3

Updated to issue a General::stop message Maybe the following does what you want? handler[Hold[Message[Test::message], True]] := With[ {stop = Flatten@Position[$MessageList, HoldForm[General::stop]]}, Which[ FreeQ[$MessageList[[stop-1]], HoldForm[Test::message]], CellPrint @ Cell[ BoxData[Internal`MessageTemplate[Test, "...


3

Here goes the benchmark for a high-end surface book with i7-6600U (2.6 up to 3.4 GHz, 4 MB cache, 15 W), which might be not quite different from surface pro 4 with intel i7-6650U (2.2 up to 3.4 GHz, 4 MB cache, 15 W). Plugged On battery


3

This issue has been fixed in 10.1+


3

Probably Child Test Files can help in your case. link


3

I had written a small package allowing to perform unit testing independently of Workbench, while also in consistency with Workbench .mt test files It looks like this: Warning: I'm not sure if Workbench uses the same .mt format. It's certainly similar but I'd appreciate comments on that. (An example of .mt file is included in the package. It's not in ...


Only top voted, non community-wiki answers of a minimum length are eligible