Smoothing a function after interpolation + trying to fit 3D data into model without accurate prior knowledge

Question

I have a physical problem simulation that generates this data set in the two cylindrical coordinates $(r,z)$ (doesn't include $\phi$ dependence). The data set (see .wdx file) is in the flattened form $(r, z, \textrm{value})$. The set represents the 3D field inside a cavity, basically a function or coordinates $E_{r}(r,z)$. We only know that roughly the dependence of $E_{r}$ on radius $r$ should be some Bessel function of the first kind in low orders, such as $aJ_{0}[k r]$ or $aJ_{1}[k r]$, or their first derivative with respect to their argument, such as $aJ'_{0}[k r]$ or $aJ'_{1}[k r]$. And the dependence on $z$ should be a sinusoidal function, such as $b\cos[d z]$ or $b\sin[d z]$. We don't know the parameters $a,k,b,d$.

Two questions:

(1) With this little prior knowledge, how can we find a model that fits this field in 3D using Mathematica? Naturally, the model should match the data at every cross section in $r$ or $z$. I have failed in finding a model using Fit, FindFit and NonlinearModelFit (but maybe because I am a novice in Mathematica). Is there a procedure that can fit arbitrary functions with little or no prior knowledge/hints given to Mathematica?

(2) I interpolated the data and got a fairly close file see .wdx file. However, I needed to calculate the partial derivative $(\partial/\partial z)$ of the interpolated field. The derivative comes out overall correctly, but its local shape is choppy and wiggly (see below). How can I smooth out this curve? Note that I have tried the Method of "Splines" but it gave worse results than the default/Hermite one.

InterpEr=Interpolation[DataEr]
Erp1=Plot[InterpEr[0.01,zz],{zz,0.0,1.6},PlotStyle->{Dashed,Black}];
Erp2=Plot[100*GetFields[1,zz,0.01][[1]],{zz,0.0,1.6},PlotStyle->Green];
Show[Erp2,Erp1]

This is the data closely fitted by the Interpolation "InterpEr" (drawn along $z$ for a given $r$ value):

zDerivativeInterpEr=Derivative[0,1][InterpEr]
Plot[{InterpEr[0.01,zz],zDerivativeInterpEr[0.01,zz]},{zz,0.0,1.6},PlotStyle->{Dashed,Black},ImageSize->Large,PlotRange->All]

This is the resulting (wiggly) derivative produced at the same instance in $r$:

I understand this wiggly shape may be a result of the data resolution due to meshing, etc, from simulation. But it would nice if I could extract the smooth shape somehow.

Answer 1

This is a quick example of smoothing the data in both the r and z directions.

Import the data and look at the points

SetDirectory[NotebookDirectory[]];
fn = FileNames["*.wdx"];
a = Import[fn[[1]]];
Dimensions[a]
{r1, r2} = MinMax[a[[All, 1]]];
{z1, z2} = MinMax[a[[All, 2]]];
Graphics3D[{Point[a]}, Axes -> True, AxesLabel -> {"r", "z", "f"}, 
 BoxRatios -> {r2 - r1, z2 - z1, 0.5}]

The data has jumps in the r direction. This is making me hesitate in doing the smoothing. Looking at sections through the data. (I am assuming the data is on a regular grid?

 zz = Nearest[a[[All, 2]] -> "Index", 0.6];
    ListPlot[a[[zz]][[All, {1, 3}]], PlotRange -> All]
    rr = Nearest[a[[All, 1]] -> "Index", 0.4];
    ListPlot[a[[rr]][[All, {2, 3}]], PlotRange -> All]

Smoothing in the r direction will smooth the jumps. Is this acceptable? In the z direction there are points out of place. Is this noise or do you need a higher resolution to get complicated details?

I am now going to do classic smoothing simultaneously in both directions. This does not seem appropriate given the jumps but you need to think about this.

sgmat = SavitzkyGolayMatrix[3, 3];
a1 = Partition[a[[All, 3]], 161];
b = ListConvolve[sgmat, a1, 1];
c = Transpose[{a[[All, 1]], a[[All, 2]], Flatten@b}];
Graphics3D[{Point[c]}, Axes -> True, AxesLabel -> {"r", "z", "f"}, 
 BoxRatios -> {r2 - r1, z2 - z1, 0.5}]

[ The data is much smoother now. Look at the cross-sections.

ListPlot[{a[[zz]][[All, {1, 3}]], c[[zz]][[All, {1, 3}]]}, 
 PlotRange -> All]
ListPlot[{a[[rr]][[All, {2, 3}]], c[[rr]][[All, {2, 3}]]}, 
 PlotRange -> All]

The data is now smooth but I suspect this is not the effect you want. We can interpolate and look at the interpolated function.

ci = Interpolation[c]; Plot3D[ci[r, z], {r, r1, r2}, {z, z1, z2}, PlotRange -> All, BoxRatios -> {r2 - r1, z2 - z1, 0.5}, PlotPoints -> {100, 50}]

I am using the SavitzkyGolayMatrix because further parameters enable a gradient to be fitted. Do look this up. Overall I think you may be better off just smoothing in the z direction. You need to think how you want to treat the jumps. You can use Savitzky Golay in just one dimension.