Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I am interested in knowing if it is possible to recover intermediate results from a Table after an interruption.

Let us say I want to produce a really useful table

tt = Table[Pause[0.1*i]; i, {i, 50}]

and since Mathematica takes too long to evaluate it, I get impatient.

Is it possible to interrupt the calculation and recover what it has already calculated? I am guessing the answer is no, but would like to be proven wrong.

share|improve this question
2  
Exact duplicate of this. See also this –  Leonid Shifrin Jun 28 '12 at 12:24
    
Indeed; sorry about that. The short answer is "it can't be done if it has not been planed" then? –  chris Jun 28 '12 at 12:26
2  
@LeonidShifrin Not an exact duplicate of the first question you mention. That deals with parallel calcluations in which side effects don't work. However the poster answers this question in the statement of his. –  jVincent Jun 28 '12 at 12:40
    
@chris Looks like that. –  Leonid Shifrin Jun 28 '12 at 12:43
    
Retagged, please feel free to suggest better tags. –  István Zachar Jul 9 '12 at 14:48

4 Answers 4

up vote 15 down vote accepted

Since I was corrected and this is indeed not an exact duplicate of the parallel computations question, I will reproduce here my implementation of abortable table, to have it here on SE:

ClearAll[abortableTable];
SetAttributes[abortableTable, HoldAll];
abortableTable[expr_, iter : {_Symbol, __} ..] :=
  Module[{indices, indexedRes, sowTag, depth =  Length[Hold[iter]] - 1},
   Hold[iter] /. {sym_Symbol, __} :> sym /. Hold[syms__] :> (indices := {syms});
   indexedRes =  Replace[#, {x_} :> x] &@ Last@Reap[
      CheckAbort[Do[Sow[{expr, indices}, sowTag], iter], Null],sowTag];
   AbortProtect[
      SplitBy[indexedRes, Array[Function[x, #[[2, x]] &], {depth}]][[##,1]] & @@ 
      Table[All, {depth + 1}]
   ]];

the usage is

tt=abortableTable[Pause[0.1*i];i,{i,50}]

(*  {1,2,3,4,5,6,7,8} *)

(I aborted the computation after some time). Generally, abortableTable accepts the same iterator syntax as Table, and can work with multiple dimensions. The details on how this works can be found in the linked discussion.

EDIT

Per request, a simple abortable Map (which only maps on the first level):

abortableMap[f_, expr_] :=
 Module[{sowTag},
   Head[expr] @@ If[# === {}, #, First@#] &[
    Last[Reap[
       CheckAbort[Do[Sow[f[part], sowTag], {part, List @@ expr}],Null]]]]]

The usage is, for example:

abortableMap[(Pause[0.1*#];f[#])&,Range[10]]

(*  {f[1],f[2],f[3]}  *)

(again, I aborted manually soon after it started to compute).

share|improve this answer
    
Just a naive question: why is this not the default then? –  chris Jun 28 '12 at 15:01
3  
@Chris I think it should not be, because this abortable table has more complex behavior than Table, and this complexity probably can not be justified in most cases, from the langauge design viewpoint. For one thing, it does not produce deterministic results, because the results depend on when you abort the computation. There are other reasons as well. –  Leonid Shifrin Jun 28 '12 at 15:22
    
An interesting bit of code, and somewhat different than the original you referenced. It's like an onion, the more you peal back, the more there is, +1. –  rcollyer Jun 28 '12 at 15:25
    
@rcollyer Thanks :) It is at the bottom of that post, I added it a little later and it probably went largely unnoticed. –  Leonid Shifrin Jun 28 '12 at 15:28
    
You're right it did. I didn't bother reading past the examples. :) –  rcollyer Jun 28 '12 at 15:32

You could use side effects instead, thus writing

tt = ConstantArray[{}, 50];
Table[Pause[0.1*i]; tt[[i]] = i, {i, 50}]

If you interrupt you will find tt partially filled, and you can then finish the rest of an aborted calculation, by letting i take on the values corresponding to the unfilled places:

Table[Pause[0.1*i]; tt[[i]] = i, {i, Flatten[Position[tt, {}]]}]

Note however that since it relies on side effects, it will for example not work if you Parallelize[] it.

share|improve this answer
    
This is clever! +1 –  Eli Lansey Jun 28 '12 at 18:10

I'm sure there must be more elegant solutions than this, but if each step takes significant time, then saving the intermediate results to disk ( or better still a RAM disk or solid state disk ) shouldn't be too much of an overhead.

The benefit here is this is partially resistant to some forms of system lockup or crash.

I've used Nest but no reason this couldn't be done with Table as well.

NestWhileList[
 With[{res = #[[2]][Last@#]}, #[[2]] >> 
    "int_res_" <> ToString[Last@#]; {Append[First@#, res], #[[2]], 
    Last@# - 1}] &, {{}, Sin, 10}, Positive@Last@# & ]

In this example Sin represents a placeholder which can be exchanged for your function of choice.

Reflection suggests that extending this to allow restarting might be a wothwhile exercise.

Update

Kernel Crash Resilient Function Application

Here is a naive, but hopefully useful, stateful solution to applying a time expensive function to a set of parameters or integers that is robust to interrupts and kernel crashes and is similar in spirit to Map or Table.

The approach is to maintain an externally stored state vector of the progression through the computation. The state vector collects the list of results and holds the unevaluated parameters and applied function.

A history option can be used to store the state for each step, allowing computation to be restarted from anywhere in the sequence.

Support is provided for named execution threads.

Basic usage

AirBag[Block[{}, Pause[#]; Print@#; #] &, {1, 3, 5, 7, 5, 3, 2}]

1

3

Out[335]=$Aborted

AirBag[]

5

7

5

3

2

Out[336]={1, 3, 5, 7, 5, 3, 2}

Named execution threads

AirBag[Composition[#^2 &, Plus], {{2, 3}, {3, 3}, {1, 4}, {5, 7}, {6, 7}}, 
       History -> True, FileSpec -> "./air_bag_composite."]

{25, 36, 25, 144, 169}

Restart the named execution thread from an arbitrary step:

AirBag["./air_bag_composite.3"]

{25, 36, 25, 144, 169}

Clear@AirBag;
Options@AirBag = {FileSpec -> "./air_bag_int_res.", History -> False};

(* Process a state vector -> {results list, function, parameter list, \
iteration counter }   *)

AirBag[state_List, OptionsPattern[]] := 
 NestWhile[
  With[{res = #[[2]][Sequence @@ (First@#[[3]])]}, {Append[First@#, 
       res], #[[2]], Rest@#[[3]], Last@#} >> 
     OptionValue@FileSpec <> 
      If[OptionValue@History, ToString[Last@#], "1"]; {Append[First@#,
       res], #[[2]], Rest@#[[3]], Last@# + 1}] &, state, 
  Length@#[[3]] > 0 &]//First

(* Apply function to the integers 1 to iterations *)

AirBag[function_, iterations_Integer, opts : OptionsPattern[]] := 
 AirBag[function, Range@iterations, opts]

(* Apply function to a list of parameter values *)

AirBag[function_, parameters_List, opts : OptionsPattern[]] := 
 AirBag[{{}, function, parameters, 0}, opts]

(* Restart from a saved AirBag state *)

AirBag[fileName_String, opts : OptionsPattern[]] := 
 If[FileExistsQ@fileName, 
  With[{state = Get@fileName}, AirBag[state, opts]]]

AirBag[OptionsPattern[]] := AirBag[OptionValue@FileSpec <> "1"]
share|improve this answer

Perhaps something like this may help you.

c = Dynamic@a
a = {};
tt = Table[AppendTo[a, {i, Pause[0.1*i]}]; i, {i, 8}]

Here, most code has to do with "automatic" abortion

kill = 0;
t = CreateScheduledTask[kill = 1];
StartScheduledTask[t];
c = Dynamic@a
a = {};
tt = Table[If[kill == 1, Abort[], AppendTo[a, {i, Pause[0.1*i]}]];  i, {i, 8}]
RemoveScheduledTask[t];
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.