37
$\begingroup$

Is there any way to build a sandbox to evaluate untrusted Mathematica expressions in order to prevent them from having (malicious or accidental) harmful side effects?

Context: I'm developing a system wherein students will enter code into designated notebook cells, and my package will extract the code, evaluate it, and offer feedback. The problem is, even if I evaluate their code within a separate context (and I'm having trouble making that happen), they could still use explicit contexts to affect a different context, invoke Quit, or use filesystem manipulation functions to mess with my computer.

It seems to me that there are two aspects to this problem: isolating execution of their code from everything else the kernel is doing (like running my package), and isolating their code from everything on my computer external to Mathematica. The first might be accomplished by using a separate kernel (somehow), but I have no ideas for the second.

Wolfram must have addressed this problem while developing WebMathematica, right?

| improve this question | |
$\endgroup$
  • $\begingroup$ You could protect your machine (files, OS, etc) by sandboxing the whole Mathematica execution. As for sandboxing code "inside" Mathematica ... there are so many ways to circumvent protections that I think you will end up trying to build a CDF environment yourself $\endgroup$ – Dr. belisarius Aug 26 '15 at 2:19
  • 6
    $\begingroup$ As a follow-up to belisarius' comment: try the program sandboxie. It won't help against Quit but it will prevent your computer from being messed up by the code. Related: list-of-dangerous-functions. $\endgroup$ – Sjoerd C. de Vries Aug 26 '15 at 6:03
  • $\begingroup$ You could also run it on Linux in VirtualBox (free). It might be overkill though. Also relevant: stackoverflow.com/questions/4249063/… $\endgroup$ – Szabolcs Aug 26 '15 at 10:50
  • $\begingroup$ Somewhat related: Automatically check student answers in tests $\endgroup$ – Jens Aug 26 '15 at 16:05
  • $\begingroup$ Alas, I'm looking for something I can do entirely within Mathematica. This is for a package that students can run to self-test their code, as well as that I can run to check the students' code. (In concept, it's a little like a unit testing suite, though with the tests written by an authority other than the programmer.) $\endgroup$ – ibeatty Aug 26 '15 at 20:38
29
$\begingroup$

You should consider using the sandbox functionality. You can create a subkernel and put it in sandbox mode this way:

link = LinkLaunch[First[$CommandLine]<> " -wstp -noicon"];
LinkWrite[link, Unevaluated@EvaluatePacket[Developer`StartProtectedMode[]]];

You can then interact with this subkernel using the standard LinkWrite and LinkRead functions. If you don't mind your master kernel being sandboxed, you can even just evaluate Developer`StartProtectedMode[] there, but it disables a lot of functionality (mostly import/export and file system manipulation).

Note that sandbox mode also will only allow you to load .m/.wl files from very specific directories. You can set this in the call itself as well:

Developer`StartProtectedMode[{"Read" -> {$myPath}, "Write" -> {$myPath}, "Execute" -> {$myPath}}]

where $myPath is the path to where you store the code you wish to interact with.

| improve this answer | |
$\endgroup$
  • $\begingroup$ @Szabolcs Incidentally, this is part of the security features of CDF Player, FreeCDF files are always executed in sandbox mode. $\endgroup$ – Stefan R Aug 27 '15 at 22:06
  • $\begingroup$ Whoa, this might be exactly what I was hoping for. The help pages and guide seem thin, though. Is there any additional documentation available? $\endgroup$ – ibeatty Aug 27 '15 at 22:20
  • $\begingroup$ @ibeatty As far as I know, the Developer`StartProtectedMode and related functions are undocumented. For information on LinkRead and LinkWrite, check out tutorial/CallingSubsidiaryWolframSystemProcesses $\endgroup$ – Stefan R Aug 27 '15 at 22:24
  • 1
    $\begingroup$ Maybe it is a good idea to use this as follows. Assuming you have configured multiple kernels, put the one you want in protected mode by evaluation Developer`StartProtectedMode[], call it "SafeKernel". Next open the notebook with NotebookOpen["~/Desktop/UnsafeNotebook.nb", Evaluator -> "SafeKernel"]. Not sure if this can be abused, but it is better than the alternative of Developer`StartProtectedMode[] inside a dangerous notebook, as the notebook can have special rules for what is actually evaluated. $\endgroup$ – Jacob Akkerboom Jan 19 '17 at 13:56
16
$\begingroup$

I have been solving exactly the same problem about 2 years ago (http://community.wolfram.com/groups/-/m/t/125587?p_p_auth=aZGMz5bs). Students are uploading piece of Mathematica (Wolfram Language) code which is run by a testing script (in Mathematica) and the results are compared with a reference solution.

To prevent the students to run potentially dangerous code I am using this initialization part (maybe I have missed some functions):

ClearAttributes[{Run,RunThrough,CopyFile,DeleteFile,RenameFile,CreateDirectory,CopyDirectory,DeleteDirectory,RenameDirectory,OpenRead,OpenWrite,Read,Write,BinaryRead,BinaryWrite},Protected]
Run[___]:=$Failed;
RunThrough[___]:=$Failed;
CopyFile[___]:=$Failed;
DeleteFile[___]:=$Failed;
RenameFile[___]:=$Failed;
CreateDirectory[___]:=$Failed;
CopyDirectory[___]:=$Failed;
DeleteDirectory[___]:=$Failed;
RenameDirectory[___]:=$Failed;
OpenRead[___]:=$Failed;
OpenWrite[___]:=$Failed;
Read[___]:=$Failed;
Write[___]:=$Failed;
BinaryRead[___]:=$Failed;
BinaryWrite[___]:=$Failed;
SetAttributes[{Run,RunThrough,CopyFile,DeleteFile,RenameFile,CreateDirectory,CopyDirectory,DeleteDirectory,RenameDirectory,OpenRead,OpenWrite,Read,Write,BinaryRead,BinaryWrite},Protected]

To detect if the testing script finished correctly I am using predefined exit code (different than the default 0). Example:

Exit[123 (* some secret code *)]

This code is tested from the "top-level" bash script which runs the Mathematica. If the student uses Quit command in his solution, it will skip my Exit with exit code and the whole test will fail.

An example how to run the testing script and test the results (exit codes):

timeout -s 9 10 math -noprompt -run < ./tester.wl >./results.txt
RET=$?

# use the same "secret code"
if [ $RET -eq 123 ]
then
# OK
fi

if [ $RET -eq 137 ]
then
# TIMEOUT
fi

I am using the timeout command, because the TimeConstrained function in Mathematica worked unreliably.

| improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ I wonder if the security features the CDF player uses can be enabled in any way. There's more than these functions that can be used for harm, though I don't know which ones of those use one of these internally. $\endgroup$ – Szabolcs Aug 26 '15 at 9:34
  • 1
    $\begingroup$ This approach could work, but I share Szabolc's nervousness that it won't eliminate all possible dangers. But perhaps that's just not possible here. $\endgroup$ – ibeatty Aug 26 '15 at 21:20
  • $\begingroup$ @Szabolcs there is a list of dangerous functions that are stored in an internal text file. I don't have the reference handy right now but have posted it on here before(maybe 2-3 years ago). Presumably if you were able to find this file in the CDF directory and modify it you could enable some functions that are otherwise prevented. ...but I have never tried to do that $\endgroup$ – Mike Honeychurch Aug 26 '15 at 23:44
  • 1
    $\begingroup$ @MikeHoneychurch Your answer to this? -- Zdenek, have you also consider setting the attribute Locked, too? $\endgroup$ – Michael E2 Aug 27 '15 at 11:17
  • $\begingroup$ @MichaelE2 yes, thanks for linking to it $\endgroup$ – Mike Honeychurch Aug 27 '15 at 20:35
3
$\begingroup$

i am doing the same thing as you did，and i am using MSP module which is a security solution for webMathmatica。please refer my topic How to adapt MSPToExpression function in $PrePrint?

sandbox seems another good solution for security，mathematica online is using it。 can i ask whether you solved this by using sandbox？

| improve this answer | |
$\endgroup$
  • $\begingroup$ Sorry to be slow in replying… Missed the question earlier. Yes, running a second kernel in sandbox mode solved my problem, although it has been incredibly difficult and error-prone, primarily because of the difficulty of constructing and sending expressions to the sandbox properly wrapped in Hold or Unevaluated or the like, without accidentally evaluating something too early, and yet getting the variables I want to be replaced properly replaced. Basically, it's a huge PITA, but… it works. Eventually. With a little black magic. $\endgroup$ – ibeatty Feb 12 '16 at 13:54
  • $\begingroup$ @ibeatty congratulation! and I solved this as well by using MSP module. but I think sandbox should be more security. Will take a investigation on that. $\endgroup$ – Michael Feb 13 '16 at 12:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.