4
$\begingroup$

I am currently trying to winsorize several datasets up to 1000 datapoints in R and in Mathematica for using those in regression estimations afterwards. Following the suggestion in the question http://mathematica.stackexchange.com/questions/152936/list-with-inadequate-structure-in-the-output-of-a-defined-function, I used

Clip[grate16, Quantile[grate16, {.05, .95}]]

to winsorize my data in Mathematica. Yet, if I use the Winsorize function from the package 'DescTools' in R with the code

Winsorize(grate16, probs = c(0.05, 0.95))

for exactly the same datasets, the results differ for some observations from about the 3 decimal-digit level which leads to heavily differing end results. Cursory documentation for this function can be found here. The code (in GitHub) for this function can be found here.

Since the deviations are so large for the same datasets, I do not think this is a numerical accuracy issue. I tried differing levels of precision and accuracy in Mathematica which, however, did not improve the situation. What am I missing?

$\endgroup$
  • 1
    $\begingroup$ I have very little experience with R; I think that discrepancies are due to the implicit method used in R's quantile function; perhaps taking a look at how WinsorizedMean works, you can infer which method better suits your data and needs $\endgroup$ – user42582 Feb 3 '18 at 10:18
  • 3
    $\begingroup$ Both Mathematica and R list the same 9 methods for estimating quantiles (Quantile in Mathematica and quantile in R). The default methods just happen to be different. So you just need to set the desired method in both to get the same results. $\endgroup$ – JimB Feb 5 '18 at 15:35
  • 1
    $\begingroup$ Can you post your grate16 somewhere, so that potential answerers can experiment with it? $\endgroup$ – J. M. will be back soon Mar 18 '18 at 12:19
  • 1
    $\begingroup$ @JimB Looking at the code here Winsorize does not use Quantile by the DescTools but quantile by base. $\endgroup$ – Anton Antonov Mar 18 '18 at 22:25
  • 3
    $\begingroup$ @AntonAntonov Good observation. I see from the link you provided that quantile is called by Winsorize without specifying one the 9 types so the default (type=7 in R) is used. This is the "mode-based estimate" in Mathematica. So to match what R does the following should be used: Clip[grate16, Quantile[grate16, {.05, .95}, {{1, -1}, {0, 1}}]]. $\endgroup$ – JimB Mar 18 '18 at 23:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.