# Improving NetChain for Physical Data

I am trying to improve my NetChain to better predict physical data from the industrial standard for water/steam published by the IAPWS.

Basically, this industrial standard divides the pressure-temperature plane into different zones. Through different empirical fits in different zones, thermodynamic data is interpolated. I have outlined by working example below.

Using Mathematica's Predict[] function and the GradientBoostedTrees method, I was able to create a prediction object that matches this industrial data quite well (most prediction is within 10-15%). I chose 95% of the data to train my Predict[] object. Empirically, I find that anything less than 95% of more than 95% has larger error (due to overfitting or underfitting probably).

url = "https://raw.githubusercontent.com/dnaneet/ML/master/se/data.dat";
datasetDirty = Import[url, "Data"]; (*has some "0" regions that need removal*)
(*names={'celcius','bar','region','kJPerKg'}*)
dataset =
Select[datasetDirty, #[[3]] !=
0 &]; (*Select only those rows and columns with non zero region*)
s = Dimensions[dataset][[1]];

T = dataset[[2 ;; s, 1]]; (*temperature, C*)
P = dataset[[2 ;; s, 2]]; (*pressure, bar*)
r = dataset[[2 ;; s, 3]] /. {0 -> "r0", 1 -> "r1", 2 -> "r2",
3 -> "r3", 4 -> "r4", 5 -> "r5"}; (*region, class/label*)
h = dataset[[2 ;; s, 4]]; (*enthalpy*)

SeedRandom[123];
thermoState = Thread[{T, P}]; (*thermodynamic state*)
mldata =
RandomSample[Thread[thermoState -> h]]; s = Dimensions[mldata][[1]];
training =
mldata[[1 ;; Ceiling[0.95*s]]]; (*90%-10% works best*)
validation =
mldata[[Ceiling[0.95*s] + 1 ;; s]];



I then created a Neural Network object that does the same thing. The NetChain contains layers that were arrived at by trial and error.

(*Putting data into proper format for NN*)
mldata2 = Table[mldata[[i, 1]] -> {mldata[[i, 2]]}, {i, 1, s, 1}];
training2 = mldata2[[1 ;; Ceiling[0.95 s] ;; All]];

net = NetChain[{
128, Tanh, 32, 8, 1},
"Input" -> 2]
NNThermo1 = NetTrain[net, training2]


The percentage error between ground truth (actual data, mldata object) and the predict object (pThermo1), ground truth (mldata object) and the NN object (NNThermo1) is found. The NN prediction is not as good as that from Predict. I can tell that the NN can be improved by tweaking the layers.

errorTbl = Table[
{
(mldata[[i, 2]] - pThermo1[mldata[[i]][[1]]])*100/mldata[[i, 2]],
First[(mldata[[i, 2]] - NNThermo1[mldata[[i]][[1]]])*100/
mldata[[i, 2]]]
}, {i, 1, s, 1000}]

ListLogPlot[Abs[{errorTbl[[All, 1]], errorTbl[[All, 2]]}],
Joined -> True, Frame -> True, FrameLabel -> {"", "%Error"},
PlotLegends -> {"From Predict[]", "From NetTrain[]"},
PlotLabel -> "%Error"]


I have watched through the Neural Network videos on Wolfram U and based on that found that a combination of DotPlus[] and Tanh[] layers as used in my example above, leads to the result I have.

Is there some data transformation or different layering (or a combination) that might help? I am not a data-scientist but am a mechanical engineer with relatively new experience in this field of neural networks. It is fascinating that Mathematica allows me to work so swiftly with NNs. I just wish to make my NN prediction better :)

The (T,P,h) data clusters in interesting fashion:

subset1 = Thread[{T, h}];
c1 = FindClusters[subset1, 4];
ListPlot[c1, Frame -> True, FrameStyle -> Black,
FrameLabel -> {"Temperature [C]", "Specific Enthalpy [kJ/kg]"},
PlotStyle -> {PointSize -> 0.012}]


subset2 = Thread[{P, h}];
c2 = FindClusters[subset2, 4];
ListPlot[c2, Frame -> True, FrameStyle -> Black,
FrameLabel -> {"Pressure [bar]", "Specific Enthalpy [kJ/kg]"},
PlotStyle -> {PointSize -> 0.012}]


• Recently i'm trying to create a neural net to predict solar radiation and for my input data, the Ramp function is more efficient that Tanh. – Erick Sabino Jun 6 at 11:26
• @ErickSabino Efficient as in accurate or faster? Thanks for the input! – dearN Jun 6 at 21:47