3
$\begingroup$

Can someone tell me how you could modify this code so that it recognizes more facial expressions? ... thanks for the help.

Manipulate[
 eyeMat = {{1/(eyeRadius - pupilRadius/2), 0}, {0, 
    1/(0.15 + eyes - pupilRadius/2)}};
 If[Norm[eyeMat.(pup - eyeCenter[[left]])] < 1, 
  pupNow = pup - eyeCenter[[left]];];
 If[Norm[eyeMat.(pup - eyeCenter[[right]])] < 1, 
  pupNow = pup - eyeCenter[[right]];];
 Column[{g = 
    Graphics[{face, eye[left, eyes], eye[right, eyes], Blue, 
      pupil[left, pupNow, eyes], pupil[right, pupNow, eyes], Black, 
      browDraw[left, brows, eyes], browDraw[right, brows, eyes], 
      Inset[mouthDraw[mouth], {0, -0.5}]}, ImageSize -> {400, 450}], 
   Classify["FacialExpression", g]}],
 {{brows, {-Pi/20, 0}}, {-0.6, 0}, {0.6, 0.15}, 
  ControlPlacement -> Left}, {{eyes, 0}, -0.07, 0.07, 
  ControlPlacement -> Left, 
  ControlType -> VerticalSlider}, {{mouth, 0.15}, -0.401, 0.4, 0.01, 
  ControlPlacement -> Left, 
  ControlType -> VerticalSlider}, {{pup, {0, 0}}, Locator, 
  Appearance -> None}, 
 Initialization :> (face = Circle[{0, 0}, {1, 1.2}]; pupNow = {0, 0};
   left = 1; right = 2;
   eyeRadius = 0.18; eyeCenter = {{-0.4, 0.15}, {0.4, 0.15}};
   pupilRadius = 0.09;
   browUp = 0.25; browW = 0.2; browAng = Pi/20;
   eye[side_, eccen_] := {Black, 
     Circle[eyeCenter[[side]], {eyeRadius + 0.05, eyeRadius + eccen}]};
   pupil[side_, pup_, 
     eccen_] := {Disk[eyeCenter[[side]] + pup + {0, pup[[2]] eccen}, 
      pupilRadius + Max[0, eccen/3]], Black, 
     Disk[eyeCenter[[side]] + pup + {0, pup[[2]] eccen}, 0.03]};
   browDraw[side_, {browAng_, browLift_}, eccen_] := 
    Rotate[{Thickness[0.01], 
      Line[{{eyeCenter[[side]] + {-browW, 
           browLift + browUp + 0.5 eccen}, 
         eyeCenter[[side]] + {browW, 
           browLift + browUp + 0.5 eccen}}}]}, 2 (side - 1.5) browAng];
   mouthDraw[s_] := 
    ParametricPlot[{Cos[u], -s Sin[u]}, {u, Pi/6, Pi - Pi/6}, 
     Axes -> False, PlotStyle -> {Red, Thickness[0.02]}, 
     PlotRange -> All];)]

I would like to make it recognize more emotions than neutral, fear and sadness.

Here is a screenshot of the interface made with the code above:

this is a screenshot

$\endgroup$
  • 2
    $\begingroup$ As I mentioned in my answer to the same question on the Wolfram Community site, the built-in facial expression classifier is trained on human faces and will not work well with simple drawings. $\endgroup$ – Rohit Namjoshi Jan 5 at 17:43
  • $\begingroup$ how could I create a machine learning that allows me to input an image and output to tell me what kind of facial expression? $\endgroup$ – Pasquale Jan 5 at 19:44
  • 2
    $\begingroup$ It's usually polite to indicate where you got code from when you post it. $\endgroup$ – bill s Jan 5 at 21:00
7
$\begingroup$

Introduction

The procedure given below relies on having a function that allows the quick generation of faces. (Like ChernoffFace described in this blog post.)

From the question descriptions and OP's comments it seems that something similar to progressive learning with drawn faces is requested. The procedure below can be seen as a manual progressive learning algorithm. (See the blog post "Progressive Machine Learning Examples".)

Chernoff faces interface

Re-making the interface using the package ChernoffFaces.m:

Import["https://raw.githubusercontent.com/antononcube/\
MathematicaForPrediction/master/ChernoffFaces.m"]

Manipulate[
 Column[{g = 
    ChernoffFace[<|"ForeheadShape" -> face[[1]], 
      "EyesVerticalPosition" -> face[[2]], "EyeSize" -> eyes[[1]], 
      "EyeSlant" -> eyes[[2]], "LeftEyebrowSlant" -> brows[[2]], 
      "LeftEyebrowTrim" -> brows[[1]], "MouthSmile" -> mouth[[2]], 
      "MouthWidth" -> mouth[[1]]|>, ImageSize -> {450, 450}], 
   Classify["FacialExpression", g]}],
 {{face, {0.5, 0.5}}, {0, 0}, {1, 1}, ControlPlacement -> Left},
 {{eyes, {0.5, 0.5}}, {0, 0}, {1, 1}, ControlPlacement -> Left},
 {{brows, {0.5, 0.5}}, {0, 0}, {1, 1}, ControlPlacement -> Left},
 {{mouth, {0.5, 0.5}}, {0, 0}, {1, 1}, ControlPlacement -> Left}
]

enter image description here

Proposed procedure

  1. Generate a relatively small set of random faces. (Say 100.)

    • Consider generating both training and testing sets, but probably the testing set is not needed because we can always evaluate the classifiers "at glance" with small random sets of faces "made on the spot."
  2. Classify the generated faces.

    • At first iteration use Classify["FacialExpression"].
  3. Evaluate the classification results.

    • If satisfactory declare success.
  4. Remove images from the set that are given wrong class labels.

    • Instead of removing the images re-label them with more appropriate labels. In this way new class labels can be introduced.
  5. Make a classifier with the reduced set of images and labels.

    • Call it newFaceExpressionClassifier.

    • Accumulate the training set.

  6. Go to step 1.

Example code

(The code below illustrates the procedure above, but does not fully adhere to it...)

Generate faces:

faceProps = {"ForeheadShape", "EyesVerticalPosition", "EyeSize", 
   "LeftEyebrowSlant", "LeftEyebrowTrim", "MouthSmile", 
   "MouthWidth"};
faceVecs = RandomReal[{0, 1}, {100, 7}];

AbsoluteTiming[
 faces = ChernoffFace[AssociationThread[faceProps -> #], 
      ImageSize -> {150, 150}] & /@ faceVecs;
]    
(* {0.26201, Null} *)

faces2 = ImageCrop@*Image /@ faces;
faces3 = ImageResize[#, {120, 150}] & /@ faces2;

Classification:

AbsoluteTiming[
 faceClasses = Classify["FacialExpression", #] & /@ faces3;
 faceClasses = faceClasses /. x_Entity :> x["Name"];
]

Evaluation:

Tally[faceClasses]

(* {{"fear", 47}, {Indeterminate, 14}, {"surprise", 10}, 
   {"happiness", 4}, {"neutral", 17}, {"sadness", 7}, {"anger", 1}} *)

qInds = Map[TrueQ[# === "neutral"] &, faceClasses];
qInds = Pick[Range[Length[faces3]], qInds];
Transpose[{qInds, faceClasses[[qInds]], faces3[[qInds]]}]

enter image description here

Mark/remove wrong classifications:

incorrect = {9, 11, 19, 36, 43, 71, 87, 89, 93, 94, 100};

Train a new classifier:

newFaceExpressionClassifier = 
  Classify[Delete[Thread[faces3 -> faceClasses], List /@ incorrect]]

Evaluate new classifier:

Map[{#, newFaceExpressionClassifier[#]} &, faces3[[incorrect]]]

enter image description here

$\endgroup$
  • $\begingroup$ How can I create an interface that allows me to insert images? $\endgroup$ – Pasquale Jan 5 at 20:05
  • $\begingroup$ @Pasquale The procedure outlined in my answer kind of answers your question. The procedure relies on having a function that allows the quick generation of faces. (Like ChernoffFace.) $\endgroup$ – Anton Antonov Jan 5 at 20:55

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.