5
$\begingroup$

I have the results of a process involving FindDistribution as a Dataset of probability distributions. Attempting to plot the PDFs of these from Dataset I kept getting failures. I eventually narrowed this down to the MixtureDistributions. A minimal example follows.

ds = Dataset@<|
   1 -> MixtureDistribution[{.2, .8}, {UniformDistribution[{-.08, 3. 10^6}], 
         GammaDistribution[.5, 88 10^4]}],
   2 -> ExponentialDistribution[1 10^-6]
   |>

Attempt to create a Dataset of PDF plots from Dataset Query fails.

ds[All,
 With[{pdf = PDF[#, x]},
   Plot[pdf, {x, 0, Quantile[#, .95]}]
   ] &
 ]

Mathematica graphics

Attempt to create list of PDF plots outside of Dataset Query succeeds.

With[{pdf = PDF[ds[#], x]},
   Plot[pdf, {x, 0, Quantile[ds[#], .95]}]
   ] & /@ Range[2]

Mathematica graphics

Quantile succeeds in the query,

ds[All, Quantile[#, .95] &]

Mathematica graphics

but PDF fails,

ds[All, PDF[#, x] &]

Mathematica graphics

This was narrowed to the MixtureDistribution (first row in Dataset and first in list).

ds[#, PDF[#, x] &] & /@ Range[2]

Mathematica graphics

Remove the MixtureDistribution and the Query succeeds.

ds2 = Dataset@<|
   1 -> NormalDistribution[],
   2 -> ExponentialDistribution[1 10^-6]
   |>;

ds2[All,
 With[{pdf = PDF[#, x]},
   Plot[pdf, {x, 0, Quantile[#, .95]}]
   ] &
 ]

Mathematica graphics

May someone confirm.
Mma 11.1 , Win 7 Ent

CASE: 3866316

$\endgroup$
  • 2
    $\begingroup$ Same on Linux; workaround Dataset[Map[PDF[#, x] &, ds // Normal]] ? $\endgroup$ – b.gates.you.know.what Mar 20 '17 at 15:54
  • 1
    $\begingroup$ Reported to WRI. $\endgroup$ – Edmund Mar 20 '17 at 15:58
4
$\begingroup$

There are three possible solutions imo to the problem:

General Advice: Use Datatset to display results only

This was already mentioned by @b.gatessucks in his comment and in (87360) gives some deeper insights into why working on Dataset can sometimes fail unexpectedly. So making ds above a simple Association will avoid the problem.

As a general recommendation I find it rather robust to avoid Dataset for processing data and only use it at the very end to (nicely) present the results. So instead of using ds[ operator1, operator2, ...] I will use ds // Query[ operator1, operator2, ...] which will work on pretty much anything.

Edit: Unfortunately in this case the problem is numerical so that only solutions B and C offer a cure. So the former "Solution A" had to be downgraded to general advice.

Solution B: Examine the error message and work on it

The error message shown is related to Reduce and the message mentioned is Reduce::ratnz. The documentation will tell you that this has to do with inexact numbers. So making MixtureDistribution use exact numbers (Rationalize) will work:

dsExact = Dataset @ Association[ 
    1 -> Rationalize @ MixtureDistribution[ 
             {.2, .8}, 
             { UniformDistribution[{-.08, 3. 10^6}], GammaDistribution[.5, 88 10^4]}
         ],
    2 -> ExponentialDistribution[1 10^-6]
]

dsExact[ All, With[
       {pdf = PDF[#, x]}, Plot[ pdf, {x, 0, Quantile[#, .95]} ]
   ] &
]

AOK

Solution C: Datasets are very easily 'upset' - so be quiet

ds[ All, Quiet @ With[
       {pdf = PDF[#, x]}, Plot[ pdf, {x, 0, Quantile[#, .95]} ]
    ] &
]

AOK

This will work with the OP's ds without any problem. So Dataset gets upset easily by error measages within.

Edit

Since Edmund has given a reasonable objection to "quieting" everything (one never knows what) I should mention, that we may as well only quiet the specific message that we do not care about, so

Off[ Reduce::ratnz ]

ds[ All, ... ]

will work as well.

Edit2:

Another way to deal with failure messages is to use the option FailureAction. So instead of using Quietwe may have done:

ds[All,
   With[ {pdf = PDF[#, x]},
      Plot[pdf, {x, 0, Quantile[#, .95]}]
   ] &,
   FailureAction -> None 
]
$\endgroup$
  • $\begingroup$ Thanks for the investigations (+1). Solution A is an interesting approach. Solution B does not work in the real case as the parameter values are not so well behaved as in the minimal working example I provided, the actual distributions are a result of FindDistribution. I'm not too keen on solution C as you never know what you are shushing and it can lead to hard to trace errors. I think I might use Solution A in this particular case but I like the syntactical sugar of auto-calling Query with Dataset so I don't see myself dropping that any time soon. $\endgroup$ – Edmund Mar 20 '17 at 17:38
  • $\begingroup$ @Edmund I have edited my answer with regard to the very reasonable objection against Solution C. Using Off we very well know what we are "shushing". $\endgroup$ – gwr Mar 20 '17 at 17:47
  • 1
    $\begingroup$ A heads up that solution A gives the same failure result as the Dataset Query: Normal@ds // Query[All, With[{pdf = PDF[#, x]}, Plot[pdf, {x, 0, Quantile[#, .95]}]] &] . Currently I'm just mapping and converting like @b.gatessucks comment in the OP. $\endgroup$ – Edmund Mar 20 '17 at 19:31
  • $\begingroup$ @Edmund Ok, I must admit, that I had not checked this more thorouhgly - but it should serve as a general remark in this case. So it is just Solution C : Off[ Reduce::ratnz ] which will make this work out. $\endgroup$ – gwr Mar 20 '17 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.