This question is a follow-up from here. I have a function that generates a list of correlations between some random variables:

varparams = Table[i, {i, 0.1, 0.9, 0.1}];

var[i_] := 
 RandomChoice[{varparams[[i]], 1 - varparams[[i]]} -> {1, 0}, 10]

corrcheck[i_, j_, n_] := 
    Correlation[var[i], var[j]]], {k, n}]

Now, if by chance one or both of var[i],var[j] is a list of all 1's, then Correlation[var[i],var[j]] is undefined. I want to improve corrcheck[i_, j_, n_] so that it automatically throws away all such vars. My idea was to do something like this:

corrcheck[i_, j_, n_] := 
   If[Mean@var[i] != 1 && Mean@var[j] != 1, 
    Correlation[var[i], var[j]], "N/A"]], {k, n}]

Why doesn't this work? Have I missed a better alternative?


A minor issue with your current test is that it ignores the case where Mean[var[i]]=0. You need to use Variance[var[i]] in your condition.

However, this does not fix the real problem. The real issue with the current formulation is that randomization is performed everytime var[i] is used. Because of this, when you check Variance[var[i]] !=0 and then later use var[i] in Correlation[var[i]],var[j]] there are actually two different var[i]s, and sometimes the first one passes your check but the second one inside Correlation has zero standard deviation. To fix this you need to use something like

 corrcheck[i_, j_, n_] := 
 Table[BlockRandom[SeedRandom[k]; With[{v1 = var[i], v2 = var[j]},
 If[Variance@v1 != 0 && Variance@v2 != 0, Correlation[v1, v2], "N/A"]]], {k, n}]

where in each iteration one random value is generated to be used for checking non-zero variance and computing the correlation for each var[i].

EDIT: Few more alternative ways to define corrcheck

ClearAll[corrcheck2, corrcheck3];
corrcheck2[i_, j_, n_] := Table[BlockRandom[SeedRandom[k]; 
    Module[{v1, v2}, If[Variance@v1 != 0 && Variance@v2 != 0, Correlation[v1, v2], 
   "N/A"] /. {v1 -> var[i], v2 -> var[j]}]], {k, n}];
corrcheck3[i_, j_, n_] :=   Table[BlockRandom[SeedRandom[k]; 
  If[Variance@#1 != 0 && Variance@#2 != 0, Correlation[#1, #2], 
   "N/A"] &[var[i], var[j]]], {k, n}];
corrcheck[1, 2, 10]
(* {"N/A","N/A",-0.21821789,"N/A",0.21821789,-0.408248290,
corrcheck2[1, 2, 10] == corrcheck3[1, 2, 10] == corrcheck[1, 2, 10]
(* True *)

EDIT: Cleanest approach is to make use of Correlation's internal testing for zero standard deviation -- using Check as in Mark's answer but placing it just outside Correlation:

corrcheckX[i_, j_, n_] := Table[BlockRandom[SeedRandom[k];
 Quiet[ Check[Correlation[var[i], var[j]], "N/A", Correlation::zerosd]]], {k, n}];
corrcheck2[1, 2, 10]==corrcheck3[1, 2, 10]==corrcheck[1, 2, 10] == corrcheckX[1, 2, 10]
(* True *)
  • $\begingroup$ Okay, now we're cooking, but (correct me if I misunderstand your code) it seems I would have to ClearAll[corrcheck] every time I want to call `corrcheck[i,j,n]. Is there a way to include ClearAll in the function itself? $\endgroup$ – Ooku Sep 27 '12 at 22:55
  • $\begingroup$ @Ooku, you don't need to use ClearAll[...] everytime you use corrcheck[i,j,n]. You need to use clear only once before you define corrcheck - just in case you previously used the symbol corrcheck with other specifications during the current mma session. $\endgroup$ – kglr Sep 27 '12 at 23:00
  • $\begingroup$ Oh, right. Thanks a lot for the prompt reply, btw! $\endgroup$ – Ooku Sep 27 '12 at 23:05
  • $\begingroup$ @Ooku, my pleasure. Welcome to Mma.SE. $\endgroup$ – kglr Sep 27 '12 at 23:09

I guess you could use Check to check for the error message and return some symbol or string indicating the failure

corrcheck[i_, j_, n_] := Quiet[Check[
  Correlation[var[i], var[j]]], {k, n}],


result = {corrcheck[8,8,1], corrcheck[8,8,2]} // InputForm
(* Out: {{-(1/Sqrt[21])}, "bad"} *)

Afterwards, you could call DeleteCases[result, "bad"] to delete the bad stuff.

  • $\begingroup$ That works after a fashion, but it maps entire sets of correlations -> bad, instead of just one of the set. $\endgroup$ – Ooku Sep 27 '12 at 16:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.