What do the numerical values obtained from “FeatureExtraction” mean and what are their applications?

I started a self study on machine learning with mathematica recently. I have seen the power of predict and classify. I am particularly impressed with "FeatureExtraction" function but I really want to know what the resulting numerical values mean. Secondly, I have a feeling these values could be used for some other analyses but I wish to know such applications from experts. Your opinions and suggestions would be greatly appreciated. Compliments of the season to you guys!!

1 Answer

To extract features means to take the original data and convert it into a different coordinate system, called a feature space, with the property that Euclidean distance can be used to check how similar objects in this coordinate system are in some undefined but desired sense.

For example, let's think about how to compare grayscale images. We could take all the pixel values in image 1 and subtract all the corresponding pixel values in image 2, and then take the norm of that vector. The smaller the norm, the more similar the images can be considered to be. Unfortunately, a white dog will be considered more similar to a white cat than a white cat to a black cat. The problem that feature spaces solve is that in machine learning we often want the latter.

Here's an example from the FeatureSpacePlot documentation:

As you can see, dogs are close to each other in this coordinate system, as are cats. FeatureSpacePlot is a visualization of the numeric feature vectors that FeatureExtract finds. The difference between those vectors and the ones that FeatureSpacePlot use is that the latter have been reduced to two dimensions for visualization purposes. The original feature space is much more high-dimensional, and as a consequence, dogs and cats will likely be even better separated in that space.

Nobody knows what the vectors that FeatureExtract creates mean because the feature spaces to which they belong were not designed to be intuitive, we only know that the vectors that FeatureExtract returns live inside feature spaces with the property that objects that are similar in some sense have a smaller Euclidean distance to each other than to objects that are not similar in that sense.

The blog post New in the Wolfram Language: FeatureExtraction talks about feature extraction in Mathematica at length, and it also provides details on the feature space for images was/is created.

• There is an interesting illustration of a relation between images and a corresponding 2 D features-space here, at time = 19' 30''. Of course it's a example. One can imagine all sort of kind of "feature space". – andre314 Dec 29 '18 at 21:31