# Automated testing for compatibility with older Mathematica versions

I have several packages which I actively develop and maintain. I try to stay up to date with new releases of Mathematica and usually update within a couple of months of a new version coming out. As a result, I can always be sure that my packages work with the latest version of Mathematica. However, not all users of my packages will want or even be able to update when new releases come out. I would like to support these users, but doing so is difficult when all my development and testing happens with only the latest version.

Is there an automated way of checking which version of Mathematica is required by my packages? In particular, I would like to know if I am using functionality which was only introduced in a specific version of Mathematica. Some specific examples I have encountered are:

• DeleteDuplicates only appeared in version 7.0
• SeriesCoefficient had a change in behaviour in some cases in version 7.0.
• Wavelet support was added in version 8.0

I am aware that the documentation usually says when a feature is new in a specific version, but some of my packages are quite large and it would not be feasible to manually check the reference pages for each function which is used. The NewInXXAlphabeticalListing guides would be less effort to check, but I would prefer an automated solution.

Once I know which version is required I can add a check against $VersionNumber when my package is loaded and output an warning if the version being used is to old. - ## 2 Answers The only reliable way seems to have a good set of unit test suites, and run them in earlier versions of Mathematica (I mention this here since the answer and comments mentioning this were deleted). However, having explicit rules for when functions were introduced and / or last changed, extracted from the docs, seems to me a good thing, which may help reduce some work, give hints, etc. So, in addition to the suggestions in other answers / comments (I particularly support the unit testing suggestion), the following code can be executed to extract the versioning information from the documentation: ClearAll[getVersionSince]; getVersionSince::fail = "Unable to extract version information for function 1"; getVersionSince[file_String?FileExistsQ] := With[{nb = NotebookOpen[file, Visible -> False]}, With[{res = Cases[ NotebookGet[nb], Cell[s_String, "History", ___]:> StringCases[StringTrim@s, {"New in" ~~ (Whitespace | "") ~~ d : ((DigitCharacter | ".") ..) ~~ __ ~~ "Last modified in" ~~ (Whitespace | "") ~~ m : (((DigitCharacter | ".") ..)) :> {d, m}, "New in" ~~ (Whitespace | "") ~~ d : ((DigitCharacter | ".") ..) :> d} ], Infinity]}, NotebookClose[nb]; First@res /; res =!= {}] /; nb =!=$Failed];

getVersionSince[file_String?FileExistsQ] :=
(Message[getVersionSince::fail, Style[FileBaseName[file], Red]]; $Failed);  Here is the setup I used: $docdir =
FileNameJoin[{$InstallationDirectory, "Documentation", "English", "System", "ReferencePages", "Symbols"}];$functions = FileNames["*.nb", {$docdir}];  To produce the rules for the functions, you can use something like rules = Table[FileBaseName[f] -> getVersionSince[f],{f,$functions}]


I actually used

j = 0;
Monitor[
functionRules =
abortableTableAlt[(j++; FileBaseName[f] -> getVersionSince[f]), {f, $functions}], j]  where the abortable table function abortableTableAlt is described at the bottom of this answer. The process was time and memory-consuming, so I saved the result to a file, available from this gist. The result has this format: {"AbelianGroup" -> {"8"}, "AbortKernels" -> {"7"}, "Abort" -> {"2"}, "AbortProtect" -> {"2"}, "Abs" -> {"1"}, "AbsoluteCurrentValue" -> {"6"}, "AbsoluteDashing" -> {{"2", "6"}}, "AbsoluteFileName" -> {"7"}, << 3400 >>, "$UserName" -> {"3"},  "$Version" -> {"1"}, "$VersionNumber" -> {"2"}, "Media" -> {"1"}}


Where a single number (string) is the earliest version when the function (or at least the doc.page) was introduced, and the format like {{"2","6"}} means that it was introduced in v.2 and last modified in v.6. In total, 3412 notebooks were processed. The only function for which this code failed was KConnectedComponents.

You can import the file from the mentioned gist, and I'd recommend to use Dispatch to speed up the rule application. Alternatively, you can use Save to attach these rules to whatever function (symbol) you want. As was suggested in another answer, this can be combined with some tool to extract dependencies, which would give you a list of system symbols used in your code. One such tool was developed David Wagner way back in 1996 and desribed in his article in the Mathematica Journal. I developed a similar tool here, but it is a work in progress and contains some bugs as of now.

-
Big +1. I started to do this myself but it was too much work. I knew someone would do this, and I am not surprised it is you. :-) –  Mr.Wizard Feb 20 '12 at 13:33
Very nice, +1. But a caveat: as I and @Mr.Wizard discovered recently, this information is not always accurate. Sometimes functions change without mention in the documentation. Sometimes they change even between minor versions (e.g. the coordinate system used by Radon (the function on which ImageLines depends) changed between 8.0.0 and 8.0.1) –  Szabolcs Feb 20 '12 at 13:38
@Mr.Wizard Thanks! Somehow I feel that the code is not optimal (it took quite a bit of time to process the docs), but I didn't have time to optimize, just left it alone and grabbed the results when it was done. –  Leonid Shifrin Feb 20 '12 at 13:38
@Leonid: I don't think that optimizing that part of the code is very important, it just needs to be run once per version. Having that list is a very useful achievement. –  Albert Retey Feb 20 '12 at 13:53
@barrywardell: I still think it is worth mentioning that that information (and additional functionality to scan packages) still can only provide a guess whether a package could possibily work with a certain version. Running tests with older versions is the only reliable way to check which version a package will work with, and automated unit tests are probably the best way to do that. I don't understand why the earlier comments to the question have been deleted, the information there seemed to be valuable. –  Albert Retey Feb 20 '12 at 13:54

Here is a way to see what is new or changed relative to an older version. Assuming English documentation,

newSince[oldversion_] := Module[{new},
new[version_] :=
StringReplace[Cases[
Import[FileNameJoin[{$InstallationDirectory, "Documentation", "English", "System", "Guides", "NewIn" <> ToString@version <> "0AlphabeticalListing.nb"}]], x_String /; StringMatchQ[x, "paclet:ref/" ~~ __], \[Infinity] ], "paclet:ref/" -> ""]; Union[Flatten[new /@ Range[oldversion + 1, Round@$VersionNumber]]]
]


You could check your notebook source file for occurrences of strings that are members of the string list returned by newSince[]:

Clear[notebookCompatibility];
notebookCompatibility[notebookFilename_, oldversion_: 6] :=
Module[{file, newlist, words},
file = Import[notebookFilename, "Text"];
newlist = newSince[oldversion];
Cases[Union@StringSplit[file, {Whitespace, "\"", "\\"}],
x_String /; MemberQ[newlist, x]]
]


I ran this on a recent notebook, and got

notebookCompatibility[<path to notebook>]

(*  {"TextRecognize"}  *)


You could remove functions from the list, depending on whether they give you problems in practice or not.

-