Removing Presenter From Slides

Question

Often you can find slide presentation where the presenter is also shown on the same slide:

When no pdf slides are available, I sometimes take scree shots of the video. However, I would like to remove the presenter, if possible.

In Mathematica there is the FindFaces[] function. It could be used to identify the person, but then I don't know how to cut it out accurately... ?

The result should look like this:

Do you know how I could do that ? ... and if not in Mathematica, do you know any other software?

Answer 1

This is an entertaining question. We'll use some neural networks from the neural network repository to attempt to solve it.

We'll use Ademxapp model, so here's a function to evaluate the net and give us back masks for each type of object it detects.

net = NetModel["Ademxapp Model A1 Trained on ADE20K Data"];

netevaluate[img_, device_: "CPU"] := 
Block[{resized, encData, dec, mean, var, prob}, 
  resized = ImageResize[img, {504}];
  encData = Normal@NetExtract[net, "Input"];
  dec = NetExtract[net, "Output"];
  {mean, var} = Lookup[encData, {"MeanImage", "VarianceImage"}];
  prob = NetReplacePart[
     net, {"Input" -> 
       NetEncoder[{"Image", ImageDimensions@resized, 
         "MeanImage" -> mean, "VarianceImage" -> var}], 
      "Output" -> Automatic}][resized, TargetDevice -> device];
  prob = ArrayResample[prob, Append[Reverse@ImageDimensions@img, 150]];
  dec[prob]]

Now we'll write a function to only get data about people in the image. Now, from the documentation in the repository, I know that the label for the "person" mask is 13, and that's the only mask that we care about.

getPeople[i_] := Map[ReplaceAll[{13 -> 1, _ -> 0}], netevaluate[i], {2}] // Image

Now we can simply get the largest item in that mask and remove it. We could try Inpaint but it didn't work very well on this image.

removePresenter[i_] := ImageAdd[i, Dilation[SelectComponents[getPeople[i], "Count", -1], 3]]

Try playing with the argument to Dilation if it doesn't take enough of the presenter. I would also consider changing the neural network for others in the "Semantic Segmentation" section if this one isn't accurate enough for you.