# How to segment rectangular photographs on a table?

Given an image of a table with photographs (take from any angle), I'm trying to extract the photographs and correctly align them into perfect rectangles. There are four steps to the algorithm:

1. Detect quadrangles in the input image

3. Re-project the quadrangles to a bird's-eye view

4. Correct for curved edges in the sub-images

Here's an example of doing the first two steps:

• For each of detected photographic quadrangles an orange outline is drawn.
• I've added a green one to emphasize the fidelity of solution I'm looking for, we can't lose any pixels in this process.
• Notice how the orange rectangles do not contain any white margins - we just want them to shrink-wrap the photographic content.

Ok, then here's what the remaining steps look like:

Note that for each of reprojected photograph (shown on the left) we fix the warping/curvature of the photograph by transforming it into a perfectly rectangular image (on the right).

Here's an example image to try:

Import @ CloudObject[
"https://www.wolframcloud.com/objects/d3999fcf-7a01-4d60-bb9c-38604d252475"]


And as requested, some additional examples to test with:

CloudGet @ CloudObject["https://www.wolframcloud.com/objects/068cf32f-b753-4c38-bc01-f378bc64d54a"]


• See ImagePerspectiveTransformation. – David G. Stork Aug 16 '18 at 23:46
• @DavidG.Stork I think ImagePerspectiveTransformation will help to re-project the image but we would have to assume a fixed aspect (the most common print aspects are 5x7, 4x6, 8x10). But deriving correct segmentations in step 1 and 2 is the hardest part IMO – M.R. Aug 16 '18 at 23:49
• Your test image is a harder version of the problem than the first image in your post ;) The lighting is much harder, the background is not a uniform color, some photos are cut off by the border. – Carl Lange Aug 24 '18 at 10:29
• @M.R. Doesn't look like your question is updated with them. I reiterate that the first image in your question is about 20 times easier to do this for than the image you post as an example at the end of your question. – Carl Lange Jan 17 '19 at 9:07
• @CarlLange just added them! – M.R. Jan 17 '19 at 19:16