0
$\begingroup$

How do I use NetChain to obtain a fit that is at least accurate on the training data?

I am trying to get a hopefully predictive fit for some data (see my question Fitting smooth monotonic function (low number of points, irregular grid)). But in the process I found I was already getting stumped on something much more elementary. I couldn't even managed to get a (possibly over-fitted) fit that actually goes close to the points in the training data.

What do I need to do adjust to obtain a fit that is at least accurate on the training data? (Should I add more layers, change the training termination criteria...?)

data={{3.38, 1.028877662, 2.009398505, 2.067322478, 4.214191194}, {3.4, 
  1.030082372, 1.995543604, 2.105894366, 4.234656059}, {3.5, 
  1.035994874, 1.992385102, 2.200815333, 4.282937808}, {3.57, 
  1.036731784, 1.986961442, 2.224357922, 4.307824219}, {3.6, 
  1.036978228, 1.985081926, 2.231988058, 4.315914728}, {3.62, 
  1.037229736, 1.983076125, 2.239730469, 4.323988127}, {3.78, 
  1.038461995, 1.969909372, 2.283628754, 4.374960036}, {3.8, 
  1.038741973, 1.96716995, 2.291334094, 4.384253554}}

Example try (I tried different things with more or less layers or different activation functions too):

net = NetChain[{30, Tanh, 30, Tanh, 30, Tanh, 30, Tanh, 30, 1}, "Input" -> "Scalar", 
  "Output" -> "Scalar"]
trainingSet = (#1 -> #3 & @@@ data);
NetInitialize[net];
net = NetTrain[net, trainingSet];

Example fit:

enter image description here

I would like to have a fit that deviates from the data in the training set at most by say 10^-15. (The data is exact up to 30 digits and so I would like a fit to reproduce it to high accuracy.)

P.S. The data as posted here is obviously only accurate to at most 9 digits but that is just the truncation I used here to keep the post readable. I don't think the error introduced by this rounding will pose any problem to finding a fit but if for some method it does I can provide the high precision data.

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ Why are you using neural networks for fitting 7 points? @BobHanlon showed you in his answer how to do a polynomial fit. Also note that there are infinitely many possible functions which go through these 7 points. Getting an "accurate" fit depends on what this data actually represent and what the underlying function is supposed to be. $\endgroup$
    – Domen
    Commented Jul 31, 2023 at 17:03
  • $\begingroup$ Right, might not be the right tool. Still it puzzles me that I can't even get it to (over)fit. Hence this question. $\endgroup$
    – Kvothe
    Commented Jul 31, 2023 at 17:36
  • $\begingroup$ Rest of my reply is more related to the question you reference. Sure there are infinitely many functions but perhaps I wasn't clear in the other question but I still expect something like occam's razor to hold. The parts of these functions seem pretty well approximated by linear and cubic polynomials as long as I split the function up. As a whole it clearly needs a much higher order polynomial. Note that BobHanlon's suggested fit is absolutely terrible FindFormula is just using linear fits which clearly don't fit at all. (O it was, just updated so I will have to check again.) $\endgroup$
    – Kvothe
    Commented Jul 31, 2023 at 17:37
  • $\begingroup$ I can obtain decent fits by putting a lot of effort by hand (looking what the pattern is going from a part where the function is linear to cubic or fine tuning orders of polynomials) but the whole point is to automate this process. (Yes I do not define mathematically what makes a fit good. If you want perhaps good criteria for that question would be as smooth as possible while being monotonic and going through the provided data points. (Polynomial fits can be ok when fine tuning the order (~7 in this case) but they have a tendency to break monotonicity.) $\endgroup$
    – Kvothe
    Commented Jul 31, 2023 at 17:41
  • $\begingroup$ Any suggestion (probably best made on the linked question) for a different method I should try are of course welcome. So far I have been playing with polynomial fits, constrained spline fits maximizing smoothness, and imposing monotonicity on a discrete set of points. So far the best was just a polynomial fit fine tuning the order but I want to automate that work. And polynomial fits tend not to be monotonic as this data is. $\endgroup$
    – Kvothe
    Commented Jul 31, 2023 at 17:48

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.