23
$\begingroup$

I understand that Mathematica can have some peculiarities in this area, but just in case...

I'm talking about things like measuring coverage of MUnit tests to get an idea of code quality.

$\endgroup$
31
$\begingroup$

Yes, there is a coverage tool, and it's pretty good.

First, you need to install the required packages:

PacletInstall["CodeParser"]
PacletInstall["Instrumentation"]

You will also need to have Gcov installed on your machine.

Then, you need to instrument your code. Your code should live in a plain text file (.m or .wl) and presumably it's in the standard package format. I will assume that it is in the MyPack directory.

Create a directory where the instrumented version will go. Let's call it coverage. Then you do:

baseline = CoverageInstrument["/full/path/to/MyPack "/full/path/to/coverage/MyPack"]

Next, you do whatever is needed to make sure that when you load your package, it is going to load the instrumented version. For example, add it to $Path or use PacletDirectoryLoad if you're using paclets.

Now you are ready to run your code. Presumably, this will consist of loading your package and evaluating the test suite.

{res, runData} = 
     CoverageEvaluate[
       Needs["MyPack`"];
       runMyTests[] (* or whatever way you have to run your test suite *)
     ];

We export the coverage data:

CoverageProcess[baseline, "/full/path/to/coverage/MyPack", "coverage-baseline.lcov"]
CoverageProcess[runData,  "/full/path/to/coverage/MyPack", "coverage-run.lcov"]

And we run the Gcov tools. On my machine they're in /opt/local/bin. You need to adjust this for your own machine.

SetDirectory["/full/path/to/coverage/MyPack"]

RunProcess[{"/opt/local/bin/lcov", "-a", "coverage-baseline.lcov", 
  "-a", "coverage-run.lcov", "-o", "coverage-combined.lcov"}]

RunProcess[{"/opt/local/bin/genhtml", "--num-spaces", "4", 
  "coverage-combined.lcov", "-o", "report"}]

Your report is now ready:

SystemOpen["report/index.html"]

enter image description here

$\endgroup$
2
  • $\begingroup$ Thank you @Szabolcs! I will look into it! Accepting your answer now $\endgroup$ Oct 25 at 15:02
  • $\begingroup$ Coming from a non-Mathematica user but IT guy: from your screenshot I'm missing one important metric of code coverage, which is branches covered. But it depends on you how important that metric is for you. [In case it's not as obvious branches means branches of control flow, so any if-elses or another way where different parts of the code are executed $\endgroup$
    – Hobbamok
    Oct 26 at 8:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.