# FindFaces: how to improve the results of the face recognition feature

In Mathematica 9, the FindFaces command offers an easy way to detect faces. In some experiments, I've been getting reasonable results, but I'd like to know if there are ways to improve the success rate.

As with the online help example, I thought the 1927 Solvay Conference photograph was a good place to start. You can find it on the Wikimedia here, at 3000 by 2171 pixels.

faces = FindFaces[solvay]

which returns a list of rectangles:

{{{81.5, 1105.5}, {262.5, 1286.5}}, {{133.5, 1038.5}, {231.5, 1136.5}}, ...

Using these to chop out some passport photographs:

passports = ImageTrim[solvay, #] & /@ faces

gives this:

I can get rid of the images that are too large by using the optional minimum and maximum sizes:

faces1 = FindFaces[solvay, {85, 130}];
passports1 = ImageTrim[solvay, #] & /@ faces1

But it's still not perfect. (The online help example seems to miss more people than I do, but there are no false positives, unlike my interesting collection of masonry people.

How would I improve these results? Is Mathematica's built-in function capable of better results, or is it just, well, a toy?

-
I love the first one –  belisarius Dec 20 '12 at 10:39
@belisarius angry physicist? :) –  cormullion Dec 20 '12 at 10:41
The false positives appear to be very sensitive to denoising. If you run, say, a TotalVariationFilter on the image, many of the false positives go away and new false positive appear (the new ones are all different for Method -> "Gaussian", "Laplacian", or "Poisson"). The obvious solution: apply a bunch of different filters to the image, do FindFaces on the results, and keep the faces that appear in all of them. :) –  Rahul Narain Dec 20 '12 at 11:05
Mr. Angry Physicist never goes away though. I think he looks like Freud. –  Rahul Narain Dec 20 '12 at 11:08
For those that might enjoy it: i.imgur.com/M5tCc.jpg –  rm -rf Dec 20 '12 at 19:40

The built-in function is certainly capable of different results, using the hidden options "TrainingFile" and "ScaleDecreaseFraction".

Under the hood, Mathematica uses OpenCV for face detection. The algorithm is controlled by information contained in an XML file, the one supplied with Mathematica is located here:

FileNameJoin[{$InstallationDirectory, "SystemFiles", "Data", "Haarcacades", "frontalface.xml"}] You can download other "trained classifiers" from the OpenCV GitHub repository here. As well as alternative classifiers for faces, there are also files for detecting various other features, such as eyes, ears, upperbody and so on. These files can be supplied to FindFaces with the "TrainingFile" option. Here are a couple of examples (I have copied the XML files from GitHub into my$UserDocumentsDirectory):

findfeatures[image_, file_, opts___] := Module[{features},
features = FindFaces[image,
{"TrainingFile" -> FileNameJoin[{\$UserDocumentsDirectory, file}], opts}];
Show[image, Graphics[{EdgeForm[{Cyan, Thick}], Opacity[0], Rectangle @@@ features}]]]

Find Lena's eyes:

Find Girl3's nose:

The results often include false positives, they can sometimes be improved with the "ScaleDecreaseFraction" option, which takes a real number between 0 and 1. Unfortunately I don't understand exactly what this option does, so it is a case of trial and error to find a good value.

For example, finding Tiffany's eyes worked well with a value of 0.7 (note that I'm using a classifier for individual eyes here, as opposed to the "eye pair" I used on Lena):

"ScaleDecreaseFraction" -> 0.7]

The best I could do with the Solvay Conference image was this: