8
$\begingroup$

Often you can find slide presentation where the presenter is also shown on the same slide:

enter image description here

When no pdf slides are available, I sometimes take scree shots of the video. However, I would like to remove the presenter, if possible.

In Mathematica there is the FindFaces[] function. It could be used to identify the person, but then I don't know how to cut it out accurately... ?

The result should look like this:

enter image description here

Do you know how I could do that ? ... and if not in Mathematica, do you know any other software?

$\endgroup$
11
$\begingroup$

This is an entertaining question. We'll use some neural networks from the neural network repository to attempt to solve it.

We'll use Ademxapp model, so here's a function to evaluate the net and give us back masks for each type of object it detects.

net = NetModel["Ademxapp Model A1 Trained on ADE20K Data"];

netevaluate[img_, device_: "CPU"] := 
Block[{resized, encData, dec, mean, var, prob}, 
  resized = ImageResize[img, {504}];
  encData = Normal@NetExtract[net, "Input"];
  dec = NetExtract[net, "Output"];
  {mean, var} = Lookup[encData, {"MeanImage", "VarianceImage"}];
  prob = NetReplacePart[
     net, {"Input" -> 
       NetEncoder[{"Image", ImageDimensions@resized, 
         "MeanImage" -> mean, "VarianceImage" -> var}], 
      "Output" -> Automatic}][resized, TargetDevice -> device];
  prob = ArrayResample[prob, Append[Reverse@ImageDimensions@img, 150]];
  dec[prob]]

Now we'll write a function to only get data about people in the image. Now, from the documentation in the repository, I know that the label for the "person" mask is 13, and that's the only mask that we care about.

getPeople[i_] := Map[ReplaceAll[{13 -> 1, _ -> 0}], netevaluate[i], {2}] // Image

getPeople example

Now we can simply get the largest item in that mask and remove it. We could try Inpaint but it didn't work very well on this image.

removePresenter[i_] := ImageAdd[i, Dilation[SelectComponents[getPeople[i], "Count", -1], 3]]

removepresenter example

Try playing with the argument to Dilation if it doesn't take enough of the presenter. I would also consider changing the neural network for others in the "Semantic Segmentation" section if this one isn't accurate enough for you.

$\endgroup$
  • $\begingroup$ Wow ! This answer is great !! Is it somehow possible to only download NetModel["Ademxapp Model A1 Trained on ADE20K Data"] once and not ever time that I insert an image ... $\endgroup$ – james Oct 6 '18 at 9:04
  • $\begingroup$ It should already only download once and store the network. It's just a bit of a slow network to be honest! You can try other networks from the repository if you find this one too slow. $\endgroup$ – Carl Lange Oct 6 '18 at 9:53
  • $\begingroup$ Hmm..every time I put a new image, it tells me "Downloading Content from the repository". So to me it seems that it is downloading the network every time, but I might be wrong. $\endgroup$ – james Oct 6 '18 at 9:58
  • 1
    $\begingroup$ Thanks a lot !! Now the answer is really great ! Thank you so much !! :)) $\endgroup$ – james Oct 6 '18 at 11:51
  • 1
    $\begingroup$ It works so well ! $\endgroup$ – james Oct 6 '18 at 11:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.