# Line of best fit regardless of peaks

Example data:

i0 = {0.825153,0.791539,0.776617,0.769379,0.765206,0.762562,0.760888,0.759366,0.758234,0.757329,0.756666,0.755823,0.755164,0.754557,0.754101,0.75392,0.753347,0.752726,0.752305,0.751845,0.751313,0.751043,0.750889,0.750245,0.75001,0.74946,0.749248,0.7488,0.748571,0.748113,0.747748,0.747098,0.746776,0.746219,0.74587,0.745322,0.745012,0.744523,0.744038,0.743368,0.742905,0.742316,0.741917,0.741432,0.740886,0.740412,0.740013,0.739543,0.738953,0.738534,0.738073,0.73774,0.737039,0.736371,0.735754,0.735179,0.734607,0.734307,0.733802,0.733451,0.733002,0.732672,0.732317,0.731824,0.731421,0.730847,0.730367,0.729878,0.729457,0.729105,0.728947,0.728634,0.728444,0.728017,0.72773,0.727337,0.727076,0.726626,0.726181,0.725928,0.725457,0.725396,0.725044,0.724779,0.724437,0.724044,0.723748,0.723408,0.723149,0.72285,0.722703,0.722278,0.721895,0.721788,0.721654,0.721434,0.721427,0.721037,0.720517,0.720087,0.719709,0.719228,0.719015,0.718668,0.718353,0.718117,0.717985,0.717787,0.717474,0.717343,0.717155,0.7168,0.716549,0.71627,0.7161,0.715958,0.715702,0.715279,0.71495,0.714633,0.714491,0.714191,0.713802,0.713669,0.713325,0.713239,0.712867,0.712474,0.712268,0.712054,0.711627,0.711383,0.711214,0.710649,0.710368,0.709823,0.709304,0.70883,0.708626,0.708265,0.707998,0.707709,0.707326,0.706876,0.706518,0.706145,0.705789,0.705294,0.705033,0.704466,0.704047,0.703572,0.703012,0.702515,0.702091,0.701962,0.701661,0.701123,0.700702,0.700395,0.7001,0.699921,0.699713,0.699289,0.699064,0.698715,0.698496,0.698198,0.697948,0.697744,0.697391,0.697295,0.696995,0.696845,0.696707,0.696611,0.696589,0.69634,0.696106,0.695959,0.695727,0.69549,0.695401,0.695379,0.695232,0.695,0.694807,0.69468,0.694697,0.694971,0.694868,0.69479,0.694717,0.694531,0.694473,0.705664,0.720512,0.728181,0.73238,0.727442,0.718399,0.711571,0.706863,0.703757,0.701785,0.700557,0.699941,0.699811,0.699818,0.699812,0.69997,0.700375,0.700653,0.700987,0.701463,0.70228,0.702771,0.70335,0.704034,0.704843,0.705525,0.706256,0.70703,0.707469,0.708158,0.708828,0.709429,0.709893,0.71047,0.710878,0.711242,0.711595,0.71225,0.712732,0.713023,0.713533,0.713857,0.714156,0.71473,0.714996,0.715175,0.715306,0.715526,0.715619,0.715896,0.715871,0.715774,0.715762,0.715694,0.715763,0.715816,0.715595,0.715365,0.715407,0.715237,0.714942,0.714915,0.714782,0.714512,0.714348,0.713821,0.713568,0.713334,0.713201,0.712912,0.712686,0.71231,0.711911,0.711566,0.711145,0.710717,0.710228,0.709968,0.709437,0.709073,0.708712,0.708228,0.707727,0.70713,0.706742,0.706209,0.705769,0.70529,0.704854,0.704344,0.703979,0.703629,0.70323,0.702941,0.702169,0.701658,0.701089,0.700624,0.700094,0.699713,0.69903,0.698558,0.698092,0.697643,0.697173,0.696649,0.696311,0.695854,0.695419,0.695326,0.695021,0.694647,0.694217,0.693737,0.693232,0.692928,0.692617,0.692318,0.691953,0.691588,0.691124,0.69077,0.690552,0.69014,0.690077,0.689852,0.68967,0.689432,0.689188,0.688971,0.688816,0.688511,0.688268,0.688182,0.687964,0.688027,0.687775,0.687347,0.686959,0.686581,0.686497,0.686124,0.685813,0.685566,0.685269,0.685273,0.684925,0.684579,0.684697,0.684524,0.684485,0.684339,0.684301,0.683948,0.683974,0.683786,0.683814,0.683864,0.683736,0.683871,0.683659,0.683414,0.683085,0.683044,0.682887,0.682849,0.68297,0.682944,0.682999,0.682713,0.682982,0.68326,0.683569,0.683833,0.684199,0.684595,0.685179,0.685794,0.686252,0.686821,0.68752,0.688219,0.689125,0.689639,0.690042,0.6906,0.691186,0.692081,0.692932,0.693534,0.694424,0.69518,0.695915,0.696773,0.697274,0.697926,0.698377,0.698785,0.521922}


I would like to create a linear (or not if a better fit) line of best fit to show the slope regardless of peaks. Here is a rough free hand example:

my attempt:

line = Fit[GaussianFilter[Log[i0], 20], {1, x}, x]
Show[
ListLinePlot[i0],
Plot[{line + 1.03}, {x, 0, Length[i0]}, PlotStyle -> Green],
ListLinePlot[GaussianFilter[i0, 100], PlotStyle -> Red] ,
PlotRange -> All
]


It's a little clunky having to manually +1.03, I feel there is some mathematical formula I am forgetting that could nicely solve this, or even make a better fit.

Ideally it would be for any curve with random peaks so identifying the first section won't work for more chaotic lines

I'm not sure if the line could be thought of as exponential? Any input would be great

data = Transpose@{Range[0, Length[i0] - 1], i0};


Select a portion that corresponds to the linear part you want to fit. By trial and error, It seems that the data[[10 ;; 190]] portion would work well here. Fit a line through that, and plot all data (black), the selected portion (red), and the line (blue):

Show[
ListLinePlot[data, PlotStyle -> Black],
ListLinePlot[data[[10 ;; 190]], PlotStyle -> Red],
Plot[LinearModelFit[data[[10 ;; 190]], {1, t}, t][x], {x, 0, data[[-1, 1]]}],
PlotRange -> All
]


You could also use EstimatedBackgroundto some advantage:

ListLinePlot[
{i0, EstimatedBackground[i0]},
PlotLegends -> {"data", "est. bkgd"}
]


i0 - EstimatedBackground[i0] does a reasonable job of removing the linear bias, if that's the ultimate goal:

ListLinePlot[i0 - EstimatedBackground[i0], PlotRange -> All]


• EstimatedBackground seems really nice! Is there a way to see how the function removes / calculates the 'background' ? From what I understand I can also change the level by sigma as a second argument? for the method of selecting a linear part, I was hoping to automate in some way; however I think the second method solves this, I'm just still a little curious if a Log[] method could be used. Thanks for the effort! Mar 20 '21 at 16:42