I am trying to train a net on a time series. It gets a set of points (x,y) as input and should try to predict the first point after the series. Now I use those and the calculated point to predict the next point and so on. The results then diverge pretty fast.

As solution I thought about using multiple iterations of the net in the training process. Is it possible...

1) to use identical layers in a net and such that their weights stay the same?

2) to apply the net in the loss function?

Example net:

samples = 20;

points = CloudGet[
d8f55850303b"][[200 ;; 1000]];
trdata = Partition[points, samples + 1, 
    1] /. {a : Repeated[_, samples], d_} :> Rule[{a}, d];
chain = NetChain[{ReshapeLayer[{2*samples}], 20, 2}, 
   "Input" -> {samples, 2}];
net = NetTrain[chain, trdata];

Plotting some test paths

steps = 50;
distance = 50;

predpath[dat_, n_] := 
  Nest[Append[#, net[#[[-samples ;; -1]]]] &, dat, 
    n][[samples + 2 ;;]];
path[i_, n_] := predpath[points[[i ;; i + samples - 1]], n];
  Table[path[i, 50], {i, distance - samples, Length[points] - samples,
     distance}]], ImageSize -> Large, Joined -> True]

results in enter image description here

One can see that the first couple point are quite good but it diverges fast.

  • 1
    $\begingroup$ You are more likely to get help if you could give a minimal example of your problem. $\endgroup$ – xslittlegrass Mar 1 '17 at 22:11
  • $\begingroup$ Recurrent neural networks for sequence prediction are pretty well developed nowadays. BasicRecurrentLayer (or LSTMLayer, etc.) in 11.1 could probably be adapted to do what you want. SequenceLastLayer and NetNestOperator could be used to repeatedly reapply the net to the final prediction. NetMapOperator and NetFoldOperator could also be used if your data don't fit into the RNN framework $\endgroup$ – Michael Curry Mar 29 '17 at 20:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.