8
$\begingroup$

I'd like to find the red bordered rectangles in this image:

enter image description here

Here are my example images:

examples = CloudGet @ "https://www.wolframcloud.com/obj/cb4a73b7-5108-4c46-985c-769aa1eef3aa";

It’s tricky because they can be nested. There are at least seven of them in the above image, my current code only detects one of them so far:

img = examples[[3]];
mf1 = MeanFilter[img, 5];
ht1 = HistogramTransform[mf1];
i = SelectComponents[SelectComponents[Binarize[ht1, .2], Small], Large];
bb = Last /@ ComponentMeasurements[i, "BoundingBox"] // First;
bbc = Mean /@ bb;
Show[i, Graphics[{Red, Opacity[0.5], Rectangle[bb[[1]], bb[[2]]]}]]

enter image description here

Related Questions:

$\endgroup$
7
$\begingroup$

Here's my take at the issue (some comments are inlined)

extractRectangles[i_Image?ImageQ] :=
Block[
    {dims, i0, i1, masks1, masks2, masks3, tfun, coords},
    (* downsampling (just to make it faster) *) 
    i0 = ImageResize[RemoveAlphaChannel[i], 500];
    dims = ImageDimensions[i0];
    (* smoothing *)
    i1 = MeanShiftFilter[i0, 3, .05, MaxIterations->5];
    (* color based segmentation *)
    masks1 = DominantColors[i1, All, "CoverageImage",
        MinColorDistance -> .02,
        ColorCoverage -> .02
    ];
    (* mask cleaning *)
    masks2 = FillingTransform @ Opening[#,2] & /@ masks1;
    (* mask selection *)
    masks3 = Flatten[Map[
        Values@ComponentMeasurements[
            #, "BoundingBox",
            And[#Rectangularity > .5, #Area > .005 Times @@ dims] &
        ]&, masks2], 1];

    (* scaling to the original image size *)
    tfun = ScalingTransform[ImageDimensions[i] / dims];
    coords = tfun /@ masks3;
    (* rectangles *)
    Rectangle @@@ coords
]

Compared to your attempt I used a MeanShiftFilter in place of the MeanFilter to better preserve the hard edges and DominantColors to get a more controlled colour segmentation.

Once that's done I want to have just one more utility to get consistent colouring for the rectangles

exprColor[expr_] := 
 RGBColor["#" <> IntegerString[Hash[Unevaluated[expr]], 16, 6]]

And now this is the result.

res = extractRectangles /@ examples;

MapThread[
 Function[{image, rectangles}, 
  HighlightImage[
   image, {{"Boundary", 10}, exprColor[#], #} & /@ rectangles, 
   ImageSize -> 400]
  ],
 {examples, res}
 ] // Column

The main issues I have still are:

  • it gets thrown off by the text and thus some rectangles are smaller than they should be
  • it does not deals very well with nested rectangles
  • it has no notion of overlapping rectangles

Unfortunately, I don't have much time to dig into this more in depth.

Cheers!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.