How can I ingest Mathematica / WL benchmark results posted at Mathematica Stack Exchange (MSE), like the ones posted here?


  • There are multiple answers with benchmarks posted

  • The benchmark results have a certain clear "list of rules" form



(* {"MachineName" -> "macbook-pro", "System" -> "Mac OS X x86 (64-bit)", "BenchmarkName" -> "WolframMark", "FullVersionNumber" -> "12.1.1", "Date" -> "November 26, 2020",
 "BenchmarkResult" -> 3.028, "TotalTime" -> 4.571, "Results" -> {{"Data Fitting", 0.291}, {"Digits of Pi", 0.249}, {"Discrete Fourier Transform", 0.363}, {"Eigenvalues of a Matrix", 0.297},
   {"Elementary Functions", 0.375}, {"Gamma Function", 0.356}, {"Large Integer Multiplication", 0.323}, {"Matrix Arithmetic", 0.227}, {"Matrix Multiplication", 0.218}, {"Matrix Transpose", 0.404},
   {"Numerical Integration", 0.463}, {"Polynomial Expansion", 0.068}, {"Random Number Sort", 0.449}, {"Singular Value Decomposition", 0.221}, {"Solving a Linear System", 0.267}}} *)
  • $\begingroup$ Please, if you upvoted (or inclined to do so) this question then (consider and) post benchmarks in the linked MSE discussion. And if you do post your benchmark results, please post WL code output from Benchmark[] not just screenshots. $\endgroup$ Commented Nov 26, 2020 at 19:13
  • $\begingroup$ What is "WL"? In some Mathematica format, file extension .wl? Or simply Mathematica code? Wolfram Language Packages? $\endgroup$ Commented Nov 27, 2020 at 8:24
  • $\begingroup$ @PeterMortensen 1. "What is "WL"?" -- WL stands for "Wolfram Language". I used "Mathematica" and "WL" as synonyms. 2. "Or simply Mathematica code?" -- Yes, simply Mathematica code. (The output code from Benchmark[].) $\endgroup$ Commented Nov 27, 2020 at 11:23
  • $\begingroup$ I challenge whoever downvoted to explain the reasons for the downvote. :) $\endgroup$ Commented Nov 28, 2020 at 23:29
  • $\begingroup$ Related community post: "Ingest and analyze benchmark results posted at MathematicaStackExchange". (You can get the notebook from there.) $\endgroup$ Commented Nov 1, 2021 at 15:21

1 Answer 1



Get all answers from MSE discussion in an XML object:

xmlObject = 
  Import["https://mathematica.stackexchange.com/questions/234881/benchmarking-with-mathematica-v-12-for-up-to-date-comparision-of-mathematica-acr/235384#235384", "XMLObject"];

Get code XML elements:

lsRes1 = Cases[xmlObject, XMLElement["code", {}, code_] :> code, \[Infinity]];

(Note of that at this point code strings are obtained.)

Remove Benchmark computation commands and convert the code strings into WL code:

lsRes2 = Map[(ToExpression[Map[If[StringQ[#] && SyntaxQ[#], #, Nothing] &, #]] &) @
    StringReplace[#, {"LaunchKernels[]" -> "", "BenchmarkReport[]" -> "", "Benchmark[]" -> ""}] &, lsRes1]

Select code (blocks) that are lists of rules:

lsRes3 = Select[Flatten /@ lsRes2, MatchQ[#, {_Rule ..}] && Length[#] > 1 &];


For each code block flatten the metadata and benchmark results into associations:

lsRes4 = Map[Association@ Cases[Flatten[# /.  HoldPattern["Results" -> r_] :> (Rule @@@ r)], _Rule] &, lsRes3];


Make sure all associations have the same keys:

lsRes5 = 
 Block[{lsAllKeys = Union[Flatten[Keys /@ lsRes4]]}, 
  Map[Join[AssociationThread[lsAllKeys, "NA"], #] &, lsRes4]


(Otherwise, the obtained dataset might be hard to use in some computations.)

Make a comparison dataset:

dsBenchmarks = Dataset[lsRes5];
dsBenchmarks = dsBenchmarks[SortBy[#BenchmarkResult &]]

enter image description here

Basic analysis

Summarize the dataset:


enter image description here

(Bar) plot results:

  If[VectorQ[DeleteCases[#2, _Missing | "NA"], NumericQ], 
    BarChart[#2, PlotLabel -> #1, PlotTheme -> "Detailed"], 
  ] &, 

enter image description here

Breakdown statistics

Partition benchmark results per system (or other criteria):

aRes = GroupBy[Normal@dsBenchmarks, #System &, Dataset];
aRes = Map[#[All, Select[#, NumericQ] &] &, aRes];

Separate the overall benchmark results from the benchmark component results:

aRes2 = Map[#[All, KeyTake[#, {"BenchmarkResult", "TotalTime"}] &] &, aRes];
aRes3 = Map[#[All, KeyDrop[#, {"BenchmarkResult", "TotalTime"}] &] &, aRes];

Show the overall benchmark results:


enter image description here

Show distribution statistical plots per system:

aResPlots3 = 
     PlotLabel -> Row[{"Number of benchmarks:", Spacer[2], Length[#]}], 
     ChartStyle -> 56, ImageSize -> Medium, PlotTheme -> "Detailed", 
     ChartLegends -> Keys[Normal@#[[1]]]] & /@ aRes3;
Grid[List @@@ Normal[aResPlots3], Dividers -> All, FrameStyle -> GrayLevel[0.7]]

enter image description here

(Note, that we have to keep an eye on the number of available benchmarks per system. Those numbers are too small here...)

Older plots

enter image description here

  • 1
    $\begingroup$ I’ve also come to find that the benchmark rarely uses 100% of my systems cpu to do calculations...i’ll get spikes up to 65% and a score that varies +- 0.3 $\endgroup$ Commented Nov 28, 2020 at 10:47
  • 1
    $\begingroup$ @morbo Hmm... interesting observation. I guess you can post question(s) at MSE about modifying the “standard” benchmark in a way that makes it more computationally challenging and/or how to get/include the CPU utilization. (Of course, certain algorithms just cannot be executed that effectively on certain CPU-OS configurations.) $\endgroup$ Commented Nov 28, 2020 at 12:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.