Timeline for How to use NetChain to obtain a fit that is at least accurate on the training data
Current License: CC BY-SA 4.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 30, 2023 at 3:30 | comment | added | Dropped Bass | Your NetChain fits the training data much better if you normalize the inputs. For example: net = NetChain[{BatchNormalizationLayer[], 30, Tanh, 30, Tanh, 30, Tanh, 30, Tanh, 30, 1}, "Input" -> "Scalar", "Output" -> "Scalar"]; Then, NetTrain with a low LearningRate so the net converges to an optimum. i.imgur.com/6zstnE8.png | |
| Aug 1, 2023 at 16:21 | history | edited | Kvothe | CC BY-SA 4.0 |
added 318 characters in body
|
| Aug 1, 2023 at 12:38 | comment | added | Kvothe | (Also edited the plot to include all data points since as Daniel pointed out it was a bit deceiving before.) | |
| Aug 1, 2023 at 12:35 | history | edited | Kvothe | CC BY-SA 4.0 |
edited body
|
| Aug 1, 2023 at 12:33 | comment | added | Kvothe |
@DanielHuber, what did you run exactly? NonlinearModelFit[data[[All,{1,3}]], {a1 } Exp[-a2 (x - a3)^2], {a1, a2, a3}, x] which I think is what you meant returns the warning "Failed to converge to the requested accuracy or precision within 100 iterations" and gives a terrible fit. Did you mean that?
|
|
| Jul 31, 2023 at 17:48 | comment | added | Kvothe | Any suggestion (probably best made on the linked question) for a different method I should try are of course welcome. So far I have been playing with polynomial fits, constrained spline fits maximizing smoothness, and imposing monotonicity on a discrete set of points. So far the best was just a polynomial fit fine tuning the order but I want to automate that work. And polynomial fits tend not to be monotonic as this data is. | |
| Jul 31, 2023 at 17:41 | comment | added | Kvothe | I can obtain decent fits by putting a lot of effort by hand (looking what the pattern is going from a part where the function is linear to cubic or fine tuning orders of polynomials) but the whole point is to automate this process. (Yes I do not define mathematically what makes a fit good. If you want perhaps good criteria for that question would be as smooth as possible while being monotonic and going through the provided data points. (Polynomial fits can be ok when fine tuning the order (~7 in this case) but they have a tendency to break monotonicity.) | |
| Jul 31, 2023 at 17:37 | comment | added | Kvothe | Rest of my reply is more related to the question you reference. Sure there are infinitely many functions but perhaps I wasn't clear in the other question but I still expect something like occam's razor to hold. The parts of these functions seem pretty well approximated by linear and cubic polynomials as long as I split the function up. As a whole it clearly needs a much higher order polynomial. Note that BobHanlon's suggested fit is absolutely terrible FindFormula is just using linear fits which clearly don't fit at all. (O it was, just updated so I will have to check again.) | |
| Jul 31, 2023 at 17:36 | comment | added | Kvothe | Right, might not be the right tool. Still it puzzles me that I can't even get it to (over)fit. Hence this question. | |
| Jul 31, 2023 at 17:03 | comment | added | Domen | Why are you using neural networks for fitting 7 points? @BobHanlon showed you in his answer how to do a polynomial fit. Also note that there are infinitely many possible functions which go through these 7 points. Getting an "accurate" fit depends on what this data actually represent and what the underlying function is supposed to be. | |
| Jul 31, 2023 at 16:57 | history | asked | Kvothe | CC BY-SA 4.0 |
