2
$\begingroup$

I have some data that when plotted I get an inclined graph, as described:

data = Import["p02q.dat", "Table"];

xdata = data[[All, 1]];
ydata = data[[All, 2]];

dataplot = Transpose[{xdata, ydata}];

plot2q = ListLinePlot[ dataplot, AspectRatio -> 1/GoldenRatio, 
  ImageSize -> 800, PlotRange -> All, GridLines -> Automatic, 
  InterpolationOrder -> 2, PlotMarkers -> {Automatic, 2}]

enter image description here

My intention is to get a leveled graph out of this one, like the one bellow: enter image description here

Could anyone suggest how to solve this, please? Data file can be downloaded here.

$\endgroup$
6
  • 1
    $\begingroup$ Find a linear best fit, use this to define an angle to the x-axis, generate a rotation matrix, and apply that to your data $\endgroup$
    – b3m2a1
    Commented Dec 21, 2020 at 19:36
  • $\begingroup$ This looks like a phase unwrap problem. This answer may help. $\endgroup$
    – Hugh
    Commented Dec 21, 2020 at 20:01
  • 4
    $\begingroup$ Find the best-fit linear function, $f(t)$, then subtract that from the data. (Don't rotate! What a mess! It means you'll have a non-function.) $\endgroup$ Commented Dec 21, 2020 at 20:51
  • $\begingroup$ Thank you! @David $\endgroup$
    – Florin
    Commented Dec 23, 2020 at 8:54
  • $\begingroup$ Thank you. @b3m2a1 $\endgroup$
    – Florin
    Commented Dec 23, 2020 at 8:54

2 Answers 2

3
$\begingroup$

Removing linear trend, as suggested in comments, is almost a one-liner (check out, Fit, FindFit, LinearModelFit):

data=Import["https://pastebin.com/raw/aWYk1Jba"];
lm=LinearModelFit[data,x,x];
ListLinePlot[Transpose[{data[[All,1]],lm["FitResiduals"]}],
PlotLabel->lm["BestFit"],PlotTheme->"Detailed"]

enter image description here

$\endgroup$
1
  • $\begingroup$ Thank you, Vitaliy. $\endgroup$
    – Florin
    Commented Dec 23, 2020 at 8:56
1
$\begingroup$

I different version of the workflow from Vitaliy's answer using both Quantile Regression and Least Squares fits. Note that the de-trending results are slightly different.

Get the QRMon package:

Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/MonadicProgramming/MonadicQuantileRegression.m"]

Get data:

data = Import["https://pastebin.com/raw/aWYk1Jba"];

Construct a QRMon workflow with data summarization, Quantile Regression and Least Squares fitting, and error plots:

qrObj = 
   QRMonUnit[data]⟹
    QRMonEchoDataSummary⟹
    QRMonQuantileRegression[1, 0.5, InterpolationOrder -> 1, Method -> {LinearProgramming, Method -> "InteriorPoint"}]⟹
    QRMonFit[{1, x}]⟹
    QRMonPlot⟹
    QRMonErrorPlots["RelativeErrors" -> False, PlotRange -> {-0.06, 0.16}];

enter image description here

enter image description here

enter image description here

Get the regression functions:

aFuncs = qrObj⟹QRMonTakeRegressionFunctions;
Map[Simplify[#[x]] &, aFuncs]

enter image description here

Get the errors, i.e., the signal de-trendings:

aErrors = qrObj⟹QRMonErrors["RelativeErrors" -> False]⟹QRMonTakeValue;
ListLinePlot /@ aErrors

enter image description here

$\endgroup$
2
  • $\begingroup$ Thank you very much, Anton. $\endgroup$
    – Florin
    Commented Dec 25, 2020 at 17:12
  • $\begingroup$ @Florin Welcome, good luck! $\endgroup$ Commented Dec 25, 2020 at 17:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.