41
$\begingroup$

In Mathematica 9, the FindFaces command offers an easy way to detect faces. In some experiments, I've been getting reasonable results, but I'd like to know if there are ways to improve the success rate.

As with the online help example, I thought the 1927 Solvay Conference photograph was a good place to start. You can find it on the Wikimedia here, at 3000 by 2171 pixels.

solvay = Import["http://upload.wikimedia.org/wikipedia/commons/6/6e/Solvay_conference_1927.jpg"];
faces = FindFaces[solvay]

which returns a list of rectangles:

{{{81.5, 1105.5}, {262.5, 1286.5}}, {{133.5, 1038.5}, {231.5, 1136.5}}, ...

Using these to chop out some passport photographs:

passports = ImageTrim[solvay, #] & /@ faces

gives this:

solvay faces

I can get rid of the images that are too large by using the optional minimum and maximum sizes:

faces1 = FindFaces[solvay, {85, 130}];
passports1 = ImageTrim[solvay, #] & /@ faces1

But it's still not perfect. (The online help example seems to miss more people than I do, but there are no false positives, unlike my interesting collection of masonry people.

How would I improve these results? Is Mathematica's built-in function capable of better results, or is it just, well, a toy?

Improve this question
$\endgroup$
5
  • 6
    $\begingroup$ I love the first one $\endgroup$
    – Dr. belisarius
    Dec 20, 2012 at 10:39
  • $\begingroup$ @belisarius angry physicist? :) $\endgroup$
    – cormullion
    Dec 20, 2012 at 10:41
  • 3
    $\begingroup$ The false positives appear to be very sensitive to denoising. If you run, say, a TotalVariationFilter on the image, many of the false positives go away and new false positive appear (the new ones are all different for Method -> "Gaussian", "Laplacian", or "Poisson"). The obvious solution: apply a bunch of different filters to the image, do FindFaces on the results, and keep the faces that appear in all of them. :) $\endgroup$
    – user484
    Dec 20, 2012 at 11:05
  • 1
    $\begingroup$ Mr. Angry Physicist never goes away though. I think he looks like Freud. $\endgroup$
    – user484
    Dec 20, 2012 at 11:08
  • 6
    $\begingroup$ For those that might enjoy it: i.imgur.com/M5tCc.jpg $\endgroup$
    – rm -rf ♦
    Dec 20, 2012 at 19:40

1 Answer 1

Reset to default
43
$\begingroup$

The built-in function is certainly capable of different results, using the hidden options "TrainingFile" and "ScaleDecreaseFraction".

Under the hood, Mathematica uses OpenCV for face detection. The algorithm is controlled by information contained in an XML file, the one supplied with Mathematica is located here:

FileNameJoin[{$InstallationDirectory,
 "SystemFiles", "Data", "Haarcacades", "frontalface.xml"}]

You can download other "trained classifiers" from the OpenCV GitHub repository here.

As well as alternative classifiers for faces, there are also files for detecting various other features, such as eyes, ears, upperbody and so on. These files can be supplied to FindFaces with the "TrainingFile" option. Here are a couple of examples (I have copied the XML files from GitHub into my $UserDocumentsDirectory):

findfeatures[image_, file_, opts___] := Module[{features},
  features = FindFaces[image, 
    {"TrainingFile" -> FileNameJoin[{$UserDocumentsDirectory, file}], opts}];
  Show[image, Graphics[{EdgeForm[{Cyan, Thick}], Opacity[0], Rectangle @@@ features}]]]

Find Lena's eyes:

findfeatures[ExampleData[{"TestImage", "Lena"}], "haarcascade_mcs_eyepair_big.xml"]

enter image description here

Find Girl3's nose:

findfeatures[ExampleData[{"TestImage", "Girl3"}], "haarcascade_mcs_nose.xml"]

enter image description here

The results often include false positives, they can sometimes be improved with the "ScaleDecreaseFraction" option, which takes a real number between 0 and 1. Unfortunately I don't understand exactly what this option does, so it is a case of trial and error to find a good value.

For example, finding Tiffany's eyes worked well with a value of 0.7 (note that I'm using a classifier for individual eyes here, as opposed to the "eye pair" I used on Lena):

findfeatures[ExampleData[{"TestImage", "Tiffany"}], "haarcascade_eye.xml",
  "ScaleDecreaseFraction" -> 0.7]

enter image description here

The best I could do with the Solvay Conference image was this:

findfeatures[solvay, "haarcascade_frontalface_alt2.xml", 
  "ScaleDecreaseFraction" -> 0.2]

enter image description here

There are no false positives, but the algorithm has failed to find Lorentz (possibly because of the beard?)

I suspect, as Rahul Narain commented, to get perfect results you would have to run the algorithm several times with different parameters and/or image pre-processing, and then do some analysis of the results.

Improve this answer
$\endgroup$
17
  • 2
    $\begingroup$ Excellent spelunking! $\endgroup$
    – rm -rf ♦
    Sep 13, 2013 at 21:19
  • 2
    $\begingroup$ Fascinating! There's much functionality hidden away, for some reason. $\endgroup$
    – cormullion
    Sep 13, 2013 at 21:21
  • 1
    $\begingroup$ @cormullion, yep. I'm sure WRI have their reasons for it, but it can be quite frustrating sometimes. $\endgroup$
    – Simon Woods
    Sep 13, 2013 at 21:28
  • 1
    $\begingroup$ Lorentz is surely contracted. +1 of course. $\endgroup$
    – Dr. belisarius
    Oct 24, 2014 at 22:05
  • 1
    $\begingroup$ @bobthechemist, here is the complete course: Use this, plus trial and error ;-) $\endgroup$
    – Simon Woods
    Oct 25, 2014 at 15:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.