0
$\begingroup$

Consider the 'm' matrix: Link to 'matrix.dat': https://drive.google.com/drive/folders/1COdMCCBKq85vMDj6Jrb2qqoUlE9eTk6w?usp=sharing

m = Import["matrix.dat"];

enter image description here

I would like to fit the surface to this data now and then subtract it from the input matrix:

{DIM1, DIM2} = Dimensions@m;
NEWDATA = Outer[{#1, #2, m[[#1, #2]]} &, Range[DIM1], Range[DIM2]];
NEWDATA2 = Flatten[NEWDATA, 1];
FIT = FindFit[NEWDATA2, a x^2 + b y + c, {a, b, c}, {x, y}];
FIT1 = (Evaluate[a (#1)^2 + b #2 + c /. FIT]) &;
DATAdetr = Table[FIT1[i, j], {i, 1, DIM1}, {j, 1, DIM2}];
AfterDetr = m - DATAdetr; 
ListPlot3D[AfterDetr, PlotTheme -> "Monochrome", ColorFunction -> ColorData["Rainbow"], MeshStyle -> None, Boxed -> False, Axes -> False, ImageSize -> 600]

The problem is that data is detrended also for zeros elements. How to make only non-zero data considered?

I have code in Matlab but I don't know if it works properly:

load matrix.dat

%%% cumsum the data
matrix_T = matrix';
[a,b] = find(matrix_T > 0); 

ind = sub2ind(size(matrix_T),a,b);
s = length(ind); % box size;

matrix_T_cum = zeros(size(matrix_T,1),size(matrix_T,2));

matrix_T_cum (ind) = cumsum(matrix_T(ind));

matrix_cum = matrix_T_cum';

%%%  fitting the model
[X,Y]=ndgrid(1:size(matrix_cum,1),1:size(matrix_cum,2));

fit_model=fit([X(:),Y(:)], matrix_cum(:),'poly55', 'Exclude', matrix_cum(:) == 0);
plot(fit_model, [X(:),Y(:)], matrix_cum(:), 'Exclude', matrix_cum(:) == 0) %only data in the box are considered

vector_matrix_cum_fitted =fit_model([X(:),Y(:)]);
matrix_cum_fitted = reshape(vector_matrix_cum_fitted ,size(matrix_cum,1), size(matrix_cum,2)); 
matrix_cum_detrend = matrix_cum - matrix_cum_fitted;
plot3(X(:),Y(:), matrix_cum_detrend(:))
$\endgroup$
10
  • 1
    $\begingroup$ Have you tried using the Select function to select just the non-zero elements? And do you need that before or after the "detrending" or both? $\endgroup$
    – JimB
    Mar 11, 2022 at 0:30
  • $\begingroup$ It would be best to fit without zeros (i.e. not to include them when you fiting) . $\endgroup$
    – ralph
    Mar 11, 2022 at 5:09
  • $\begingroup$ What happened when you used Select? $\endgroup$
    – JimB
    Mar 11, 2022 at 7:07
  • 2
    $\begingroup$ NEWDATA3 = Select[NEWDATA2, #[[3]] != 0 &] $\endgroup$
    – JimB
    Mar 11, 2022 at 8:09
  • 1
    $\begingroup$ Getting NEWDATA2 can be written Flatten[MapIndexed[Flatten@{#2, #1} &, m, {2}], 1] $\endgroup$
    – MikeY
    Mar 11, 2022 at 18:27

1 Answer 1

1
$\begingroup$

I think @JimB answered your question, but I'll work it through.

m = Import["matrix.dat"];
MatrixPlot@m

enter image description here

NEWDATA2 = Flatten[MapIndexed[Flatten@{#2, #1} &, m, {2}], 1];
ND3      = Select[NEWDATA2, #[[3]] != 0 &];

Plotting just the nonzero parts

ListPlot3D[ND3]

enter image description here

Get the indices of the nonzero parts

indices = ND3[[All, {1, 2}]];

Get your rules for the fit

FIT = FindFit[ND3, a x^2 + b y + c, {a, b, c}, {x, y}]

(*  {a -> 0.0541017, b -> 2.68527, c -> -86.772}  *)

Make a function fit, slightly different form

fcn[{x_, y_}] := (Evaluate[a x^2 + b y + c /. FIT]);

Get your detrended data

dtd = Map[Flatten@{#, fcn[#]} &, indices];

ListPlot3D[{ND3, dtd}]

enter image description here

To get the detrended data dtd back to a matrix, we can use a SparseArray as an intermediary. Create the rules for the sparse array.

sar = Map[{#[[1]], #[[2]]} -> #[[3]] &, dtd];
sar[[1]]
(*   {1, 42} -> 26.0632   *)

sa = SparseArray[sar]

enter image description here

And subtract this from the original matrix

MatrixPlot[m - sa]

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.