8
$\begingroup$

Often you can find slide presentation where the presenter is also shown on the same slide:

enter image description here

When no pdf slides are available, I sometimes take scree shots of the video. However, I would like to remove the presenter, if possible.

In Mathematica there is the FindFaces[] function. It could be used to identify the person, but then I don't know how to cut it out accurately... ?

The result should look like this:

enter image description here

Do you know how I could do that ? ... and if not in Mathematica, do you know any other software?

$\endgroup$

1 Answer 1

11
$\begingroup$

This is an entertaining question. We'll use some neural networks from the neural network repository to attempt to solve it.

We'll use Ademxapp model, so here's a function to evaluate the net and give us back masks for each type of object it detects.

net = NetModel["Ademxapp Model A1 Trained on ADE20K Data"];

netevaluate[img_, device_: "CPU"] := 
Block[{resized, encData, dec, mean, var, prob}, 
  resized = ImageResize[img, {504}];
  encData = Normal@NetExtract[net, "Input"];
  dec = NetExtract[net, "Output"];
  {mean, var} = Lookup[encData, {"MeanImage", "VarianceImage"}];
  prob = NetReplacePart[
     net, {"Input" -> 
       NetEncoder[{"Image", ImageDimensions@resized, 
         "MeanImage" -> mean, "VarianceImage" -> var}], 
      "Output" -> Automatic}][resized, TargetDevice -> device];
  prob = ArrayResample[prob, Append[Reverse@ImageDimensions@img, 150]];
  dec[prob]]

Now we'll write a function to only get data about people in the image. Now, from the documentation in the repository, I know that the label for the "person" mask is 13, and that's the only mask that we care about.

getPeople[i_] := Map[ReplaceAll[{13 -> 1, _ -> 0}], netevaluate[i], {2}] // Image

getPeople example

Now we can simply get the largest item in that mask and remove it. We could try Inpaint but it didn't work very well on this image.

removePresenter[i_] := ImageAdd[i, Dilation[SelectComponents[getPeople[i], "Count", -1], 3]]

removepresenter example

Try playing with the argument to Dilation if it doesn't take enough of the presenter. I would also consider changing the neural network for others in the "Semantic Segmentation" section if this one isn't accurate enough for you.

$\endgroup$
6
  • $\begingroup$ Wow ! This answer is great !! Is it somehow possible to only download NetModel["Ademxapp Model A1 Trained on ADE20K Data"] once and not ever time that I insert an image ... $\endgroup$
    – james
    Oct 6, 2018 at 9:04
  • $\begingroup$ It should already only download once and store the network. It's just a bit of a slow network to be honest! You can try other networks from the repository if you find this one too slow. $\endgroup$
    – Carl Lange
    Oct 6, 2018 at 9:53
  • $\begingroup$ Hmm..every time I put a new image, it tells me "Downloading Content from the repository". So to me it seems that it is downloading the network every time, but I might be wrong. $\endgroup$
    – james
    Oct 6, 2018 at 9:58
  • 1
    $\begingroup$ Thanks a lot !! Now the answer is really great ! Thank you so much !! :)) $\endgroup$
    – james
    Oct 6, 2018 at 11:51
  • 1
    $\begingroup$ It works so well ! $\endgroup$
    – james
    Oct 6, 2018 at 11:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.