How can I programmatically call MUnit tests in Wolfram Workbench?

Question

I have a collection of .mt test files in Wolfram Workbench. Is there a way to invoke the tests programmatically and then write the results to a text file for database? And if not .mt tests in Workbench, is there another way to accomplish the task of programmatically calling a collection of unit tests and exporting the results?

It is important that I be able to call the tests programmatically, so that the tests can be a necessary step of our automated build process.

Thanks in advance for your guidance.

Edit: By a programmatic call I mean the ability to invoke the tests from some kind of script file (which I am entirely willing to write if pointed to the right literature).

Answer 1

Wolfram Workbench uses MUnit` package to run tests. MUnit` provides TestRun function that can be used from Mathematica (Workbench doesn't use this function, it has special test runner written in Java, but that would be hard to use for what you want).

To use TestRun function, first you need to load MUnit` package. If you're using Mathematica version 10, you can simply call:

Needs["MUnit`"]

For older Mathematica versions, you would need to find path to MUnit` package inside your Workbench installation and add it to $Path. In my installation it would be something like this:

PrependTo[
    $Path,
    FileNameJoin["/path/to/eclipse/configuration/org.eclipse.osgi/bundles/497/1/.cp/MathematicaSourceVersioned/Head"]
];

Note that in some Workbench setups, MUnit` might be inside a jar file.

After loading the package you could for example redirect output from one of ...PrintLoggers to a file:

resultsStream = OpenWrite["path/to/results.txt"];
Block[{$Output = Append[$Output, resultsStream]},
    TestRun["path/to/testFile.mt", Loggers :> {VerbosePrintLogger[]}]
]
Close[resultsStream]

If you need more control over test results logging, you could write your own custom logger. Take a look at built in loggers in MUnit/Kernel/Loggers.m file to see how to do it.

Also my WWBCommon project might be of interest. It provides some tools for running tests from ant scripts.