# Can LearnDistribution learn conditionality?

I would like to apply LearnDistribution to multivariate conditional distributions. Here's a simple artificial example:

samples = Flatten /@ {
{"crm",RGBColor[0.11,0.5,0.91],RGBColor[0.5,0.8,0.8],{.5,.55,.5,1},.1}, {"crm",RGBColor[0.12,0.5,0.92],RGBColor[0.45,0.8,0.8],{.5,.5,.5,1},.1},
{"crm",RGBColor[0.14,0.5,0.9],RGBColor[0.5,0.9,0.78],{.5,0,.5,1},.1}, {"crm",RGBColor[0.1,0.5,0.91],RGBColor[0.5,0.85,0.8],{0,0.03,.48,.95},.1},
{"crm",RGBColor[0.17,0.5,0.9],RGBColor[0.2,0.723,0.74], {0,.5,.45,1},.1}, {"crm",RGBColor[0.12,0.55,0.9],RGBColor[0.1,0.69,0.77],{0.,0,.5,1},.1},
{"crm",RGBColor[0.1,0.52,0.9],RGBColor[0.15,0.7,0.7],{0,0,.5,1},.1}, {"ibm",RGBColor[1,0.5,0.65], RGBColor[.98,1,0],{0,0,.5,.5},.05},
{"ibm",RGBColor[.99,0.54,0.68], RGBColor[.98,1,0],{0,0,1,1},.049}, {"ibm",RGBColor[.96,0.5,0.61], RGBColor[.99,1,0],{0,0,.3,.3},.045},
{"ibm",RGBColor[1,0.4,0.6],RGBColor[.98,1,0],{0.1,0.1,.9,.9},.052}, {"ibm",RGBColor[.98,0.44,0.62],RGBColor[1,.94,0],{0,0.48,1,.97},.051},
{"ibm",RGBColor[.92,0.41,0.6],RGBColor[.99,.95,0],{0,0.48,1,.94},.049}, {"ibm",RGBColor[.94,0.41,0.61],RGBColor[.96,.94,0.01],{0,0.48,1,.94},.05}
} // RandomSample;


The variable samples looks like this:

Each row vector represents an abstract glyph:

{class, background color, edge color, {x, y, w, h}, edge thickness}


The glyphs come in two classes "ibm" or "crm", the former encodes squarish red glyphs with yellow edges, and the later encodes tall blue glyphs with thinner cyan edges.

viz[s_] := Multicolumn[(Labeled[Graphics[{FaceForm[#2], EdgeForm[{#3, Thickness[#8]}], Rectangle[{#4,#5}, {#6+#4,#7+#5}]}, ImageSize -> {100,100}], #1])& @@@ s, 10, Appearance -> "Horizontal"];
viz @ samples


The colors and aspect ratios are linear relationships between positional elements in the input vectors that are conditional on the first element, i.e. the class. The docs indicate that LearnDistribution should be able to model this sort of conditionality, so let's run all four applicable methods of LearnDistribution:

methods = {"Multinormal", "GaussianMixture", "KernelDensityEstimation", "DecisionTree"};
lds = Table[LearnDistribution[samples, PerformanceGoal -> "Quality", Method -> m], {m, methods}]


The mixture method seemed to learn some of the correct coloring information, but none of the methods were able to learn the latent relationships correctly:

MapThread[Labeled, {Table[
r = GroupBy[RandomVariate[ld, 8], First] // Values;
Panel[viz /@ r //Column], {ld, lds}], methods}]


Can anyone can shine some light on LearnDistribution's capabilities with regards to this type of problem?

Is this function simply not powerful enough to model mixed-type vectors where elements are conditional upon each other in simple ways like this? If not, what other approaches might work here instead given enough data?

• Was really hoping for a dev to post about the limitations and/or tricks of using LearnDistribution and if a gan based method is coming @EtienneB
– M.R.
Commented Aug 31, 2021 at 6:08
• One solution would be to submit a product improvement proposal. Commented Aug 31, 2021 at 22:58

One resource that is helpful from the Wolfram Technology Conference is https://www.conferencecast.tv/talk-17222. This resource is full of machine learning examples and in-depth discussion and analysis of how Learn Distribution works. I also tried increasing the TimeGoal to 100seconds and that helped some.

After using TimeGoal to make it work harder, I have one more idea.

It might work to increase the Max Iterations in the application of Learned Distribution to Probability Density Function and Monte Carlo methods.

Another useful thing to figuring out what is the best solution would be to use the Information on each ML function.

This also has some interesting info on LearnDistribute.

• The link you mention at the end of your post isn't clickable. MaxIterations is not an option of RandomVariate. Can you please format your text and code correctly?
– M.R.
Commented Aug 30, 2021 at 21:13
• The conference link should be clickable. It does require you to sign up but it's free to watch the whole thing. I will revise the part about MaxIterations according to the documentation: When acting on a LearnedDistribution[…], the functions PDF and RarerProbability can be used with the following options: MaxIterations Automatic number of iterations to use when a Monte Carlo integration is performed. I took a screenshot of a Mathematica notebook at the end. I hope this helps. Commented Aug 31, 2021 at 0:24
• Yes adding PerformanceGoal -> Quality or increasing TimeGoal does help a little bit, but increasing MaxIterations doesn't seem to improve results in PDF/RandomVariate/SynthesizeMissing... but this is essentially where I had already gotten to with "GaussianMixture". Any other ideas?
– M.R.
Commented Sep 2, 2021 at 2:17
• I don't know that much about machine learning in Mathematica but emailing Wolfram Support or submitting Product Feedback for details might be a good idea. Commented Sep 2, 2021 at 20:59