3
$\begingroup$

Summary:

The first order response of a block of metal quenching in a fluid (or a cold block heating in a warm environment) is well known. I am trying to create a physics based predictive model for this phenomena, using Predict[] and GradientBoostedTrees. I would like to achieve a higher accuracy if at all possible and learn a bit more on choice of Predict[] parameters.

Details:

I am trying to create a "physics based predictive model" for a first order response of a temperature measurement device, using Mathematica's Predict[] function. Once this model is trained via a data of {time, Tinit, Tamb, tau}, I would like to test it's efficacy versus exact/analytical solutions.

$$T(t) = T_{amb} + (T_{init} - T_{amb})e^{-t/\tau}$$

Clear[t, T0, t1, \[Tau], Tinit, Tamb, tc, time];
T[t_, T0_, T1_, \[Tau]_] := 
  Module[{Tinit = T0, Tamb = T1, tc = \[Tau], time = t},
   Tamb + (Tinit - Tamb) Exp[-time/tc]];

Preparing inputs:

Clear[input, nPoints, output, time, Tinit, Tamb, tc];
nPoints = 1000;
time = ## & @@@ RandomReal[{0, 100}, {nPoints, 1}];(*time in seconds*)
Tinit = ## & @@@ RandomReal[{268, 300}, {nPoints, 1}];(*initial temp, Kelvin*)
Tamb = ## & @@@ RandomReal[{288, 400}, {nPoints, 1}]; (*ambient temp, Kelvin*)
tc = ## & @@@ RandomReal[{1, 20}, {nPoints, 1}];(*time constant of measurement device*)



input = Thread[{time, Tinit, Tamb, tc}];
output = N[#3 + (#2 - #3) Exp[-#1/#4]] & @@@ input;

dataSet = Thread[input -> output];
s = Dimensions[dataSet][[1]];

I have had plenty of success with the GradientBoostedTrees model and continued to use it here.

Predictive model:

p = Predict[dataSet, 
    Method -> {"GradientBoostedTrees", "BoostingMethod" -> "Gradient",
       "MaxDepth" -> 5, MaxTrainingRounds -> 2}, 
    PerformanceGoal -> "Quality"];

The choice of hyperparameters such as MaxDepth or MaxTrainingRounds did not make the greatest of difference.

Comparison between prediction and exact physics:

First order response for 75 seconds of time:

predT = Table[p[{t, 293, 310, 15}], {t, 1, 75, 1}]; (*prediction*)
physT = Table[T[t, 293, 310, 15], {t, 1, 75, 1}]; (*exact*)

Visualization of error between prediction and exact solution:

How can I ensure that the prediction is far more accurate than the what is achieved using Predict[]

Does it come down to just choosing better hyperparameters? I have tried several permutations and combinations, including letting Predict[] decide. The prediction is pretty consistent.

Conversely, is there some good resource (besides the Help pages and experiential learning) to help choosing hyperparameters?

Although the prediction is within pretty good, physically, the transients (t=0-30 seconds) and the steady state are inaccurate. This is a rather simple physical system.

ListLogPlot[Abs[physT - predT]*100/physT, PlotStyle -> {Thick, Red}, 
 Frame -> True, FrameStyle -> Black, 
 FrameLabel -> {"Time [s]", "Error"}, PlotRange -> {All, {0, 15}}]
Show[{ListPlot[physT, Frame -> True, FrameStyle -> Black, 
   FrameLabel -> {"time", "temperature"}, 
   PlotRange -> {All, {293, 315}}], 
  ListPlot[predT, PlotStyle -> {Thick, Red}]}]

enter image description here

The red dotted line is the prediction using gradient boosted decision trees while the blue dotted line is the exact physics solution.

Update: Predict with GaussianProcess as suggested by a kind commenter, provided a rather high bad (poor quality of prediction) as seen in the error plot below:

enter image description here

$\endgroup$
  • 1
    $\begingroup$ This seems more like an interpolation sort of problem than a real machine learning problem. You might give the "GaussianProcess" method a go, since that's basically a more elaborate form of interpolation. $\endgroup$ – Sjoerd Smit Apr 6 at 18:32
  • $\begingroup$ @SjoerdSmit So the GaussianProcess method, out of the box, gave me a terrible "interpolation". The error between the prediction and the exact solution was of the order of about 10%. Have updated the answer with this detail. So what exactly is the Predict function for? Is it just interpolation? I can understand that given a dataset, using Neural Networks or such out of the box are interpolation. $\endgroup$ – dearN Apr 6 at 18:43
  • 1
    $\begingroup$ The training data uses time constants in the range {1, 10}. The comparison uses a time constant of 15, which is well outside of the training range. Is that intentional? $\endgroup$ – Rohit Namjoshi Apr 6 at 18:45
  • 2
    $\begingroup$ In machine learning, it is typically not the tuning of hyperparameters that solves the problem, or even the choice of method. More commonly, it comes down to feeding your algorithms with enough data. In this case, for example, if you up the number of examples to 50,000 and the number of training rounds to 100, you get this, which is much better. With even more examples, (and perhaps with a bit of hyperparameter tuning,) I'm sure you can get even better results. By the way, you forgot to define the T function. $\endgroup$ – C. E. Apr 6 at 19:30
  • 1
    $\begingroup$ @drN Training machine learning algorithms is supposed to take time. Training neural networks for some image processing tasks, for example, can take days on high-end GPUs. The salient feature of these algorithms, though, is that the time it takes to make a prediction does not increase with the number of input-output examples. So, while the time it takes to train the method increases with the number of examples, this is not commonly seen as a problem. $\endgroup$ – C. E. Apr 6 at 19:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.