How to visualize a random forest classifier?

Question

I'd like to recreate this visualization (from python) in Mathematica:

I'm not sure how to extract the decision boundaries from a classifier with "Method" set to "RandomForest".

Here's the code:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs
from sklearn.ensemble import RandomForestClassifier

X, y = make_blobs(centers=[[0, 0], [1, 1]], random_state=61526, n_samples=50)

def plot_forest(max_depth=1):
    plt.figure()
    ax = plt.gca()
    h = 0.02

    x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
    y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))

    if max_depth != 0:
        forest = RandomForestClassifier(n_estimators=20, max_depth=max_depth,
                                        random_state=1).fit(X, y)
        Z = forest.predict_proba(np.c_[xx.ravel(), yy.ravel()])[:, 1]
        Z = Z.reshape(xx.shape)
        ax.contourf(xx, yy, Z, alpha=.4)
        ax.set_title("max_depth = %d" % max_depth)
    else:
        ax.set_title("data set")
    ax.scatter(X[:, 0], X[:, 1], c=np.array(['b', 'r'])[y], s=60)
    ax.set_xlim(x_min, x_max)
    ax.set_ylim(y_min, y_max)
    ax.set_xticks(())
    ax.set_yticks(())


def plot_forest_interactive():
    from IPython.html.widgets import interactive, IntSlider
    slider = IntSlider(min=0, max=8, step=1, value=0)
    return interactive(plot_forest, max_depth=slider)

Answer 1

Firstly I'm not sure if Mathematica's Random Tree method has an equivalent max_depth option (I don't know too much about random tree). The options available to the Random Tree method are: "TreeNumber", "LeafSize", and "VariableSampleSize".

Now as for plotting the classifier boundaries, one can simply pass the ClassifierFunction for ContourPlot (or similar).

My mock data:

    n = 50;
    blobs = RandomReal[{0, 1}, {n, 2}];

    blobsRed = RandomSample[(Norm /@ blobs)^4 -> blobs, Round[n/2]];
    blobsBlue = Complement[blobs, blobsRed];

    training = Join[Thread[blobsRed -> Red], Thread[blobsBlue -> Blue]];

Create the ClassifierFunction:

    c = Classify[training, Method -> {"RandomForest", "LeafSize" -> 5}];

And plot:

    ContourPlot[c[{x, y}, {"Probability", Red}], {x, 0, 1}, {y, 0, 1},
     ColorFunction -> Function[z, Directive[Opacity[0.6], ColorData["Rainbow"][z]]],
     Contours -> 7,
     FrameTicks -> None,
     Method -> {"TransparentPolygonMesh" -> True},
     PlotRangePadding -> None,
     Epilog -> {
       PointSize[0.02],
       {Red, Point[blobsRed]},
       {Blue, Point[blobsBlue]}
       }
     ]

You could also classify a grid of points and then use ListContourPlot, which will be much faster, to get an image more like:

I trust you can then use ListAnimate or Manipulate to recreate the animated aspect if that's what you wanted, if not I can expand my reply to cover that.

Hope this helps!