# Fitting data in a matrix

m = Import["matrix.dat"];


I would like to fit the surface to this data now and then subtract it from the input matrix:

{DIM1, DIM2} = Dimensions@m;
NEWDATA = Outer[{#1, #2, m[[#1, #2]]} &, Range[DIM1], Range[DIM2]];
NEWDATA2 = Flatten[NEWDATA, 1];
FIT = FindFit[NEWDATA2, a x^2 + b y + c, {a, b, c}, {x, y}];
FIT1 = (Evaluate[a (#1)^2 + b #2 + c /. FIT]) &;
DATAdetr = Table[FIT1[i, j], {i, 1, DIM1}, {j, 1, DIM2}];

ListPlot3D[AfterDetr, PlotTheme -> "Monochrome", ColorFunction -> ColorData["Rainbow"], MeshStyle -> None, Boxed -> False, Axes -> False, ImageSize -> 600]


The problem is that data is detrended also for zeros elements. How to make only non-zero data considered?

I have code in Matlab but I don't know if it works properly:

load matrix.dat

%%% cumsum the data
matrix_T = matrix';
[a,b] = find(matrix_T > 0);

ind = sub2ind(size(matrix_T),a,b);
s = length(ind); % box size;

matrix_T_cum = zeros(size(matrix_T,1),size(matrix_T,2));

matrix_T_cum (ind) = cumsum(matrix_T(ind));

matrix_cum = matrix_T_cum';

%%%  fitting the model
[X,Y]=ndgrid(1:size(matrix_cum,1),1:size(matrix_cum,2));

fit_model=fit([X(:),Y(:)], matrix_cum(:),'poly55', 'Exclude', matrix_cum(:) == 0);
plot(fit_model, [X(:),Y(:)], matrix_cum(:), 'Exclude', matrix_cum(:) == 0) %only data in the box are considered

vector_matrix_cum_fitted =fit_model([X(:),Y(:)]);
matrix_cum_fitted = reshape(vector_matrix_cum_fitted ,size(matrix_cum,1), size(matrix_cum,2));
matrix_cum_detrend = matrix_cum - matrix_cum_fitted;
plot3(X(:),Y(:), matrix_cum_detrend(:))

• Have you tried using the Select function to select just the non-zero elements? And do you need that before or after the "detrending" or both?
– JimB
Mar 11, 2022 at 0:30
• It would be best to fit without zeros (i.e. not to include them when you fiting) . Mar 11, 2022 at 5:09
• What happened when you used Select?
– JimB
Mar 11, 2022 at 7:07
• NEWDATA3 = Select[NEWDATA2, #[[3]] != 0 &]
– JimB
Mar 11, 2022 at 8:09
• Getting NEWDATA2 can be written Flatten[MapIndexed[Flatten@{#2, #1} &, m, {2}], 1] Mar 11, 2022 at 18:27

m = Import["matrix.dat"];
MatrixPlot@m


NEWDATA2 = Flatten[MapIndexed[Flatten@{#2, #1} &, m, {2}], 1];
ND3      = Select[NEWDATA2, #[[3]] != 0 &];


Plotting just the nonzero parts

ListPlot3D[ND3]


Get the indices of the nonzero parts

indices = ND3[[All, {1, 2}]];


Get your rules for the fit

FIT = FindFit[ND3, a x^2 + b y + c, {a, b, c}, {x, y}]

(*  {a -> 0.0541017, b -> 2.68527, c -> -86.772}  *)


Make a function fit, slightly different form

fcn[{x_, y_}] := (Evaluate[a x^2 + b y + c /. FIT]);


dtd = Map[Flatten@{#, fcn[#]} &, indices];

ListPlot3D[{ND3, dtd}]


To get the detrended data dtd back to a matrix, we can use a SparseArray as an intermediary. Create the rules for the sparse array.

sar = Map[{#[[1]], #[[2]]} -> #[[3]] &, dtd];
sar[[1]]
(*   {1, 42} -> 26.0632   *)

sa = SparseArray[sar]


And subtract this from the original matrix

MatrixPlot[m - sa]