How to find the mark centroid

Question

This is my target img:

I can binarize it and select the mark component:

bin = Opening[ColorNegate[Binarize[img, 0.8]], 4]

Then I can find the mark centroid with ComponentMeasurements:

HighlightImage[img, Values[ComponentMeasurements[bin, "Centroid"]]]

But in this post, I want to request a method that don't binarize the img. The binarition method always has to consider the thick of those lines and the threshold about white or black. So I want to process the gray image directly? It is possible?

---

There are some series image to process:

img1:

img2:

img3:

---

As the comment discuss, I have made some test dataset here.

Answer 1

You can detect dark blobs without binarization using a Laplacian of Gaussian filter. First remove the thin lines from the image using dilation:

dilated = Dilation[img, 3]

then use an LoG filter to get high local maxima for dark blobs:

log = ImageAdjust[LaplacianGaussianFilter[dilated, {50, 10}]]

then find dark blobs:

max = MaxDetect[log, 0.15]
HighlightImage[img, max]

This seems to work for all your images:

Table[
  (
   dilated = Dilation[img, 3];
   log = ImageAdjust[LaplacianGaussianFilter[dilated, {50, 10}]];
   max = MaxDetect[log, 0.15];
   HighlightImage[img, max, ImageSize -> 450])
  ,
  {img, imgs}] // Multicolumn

However, it's not parameter-free. I'm not sure you can do this without any parameters. The best you can do is to get parameters that are as tolerant as possible to e.g. different lighting, different rotation etc.

Answer 2

Training a classifier

Let's go through the process of training a classifier for this problem. We start by loading the images:

imgs = Import /@ FileNames[".../dir1/*.png"];
imgs = Join[
   imgs,
   Import /@ FileNames[".../dir2/*.png"]
   ];
imgs = Join[
   imgs,
   Import /@ FileNames[".../dir3/*.png"]
   ];
imgs = RandomSample[imgs];

Randomizing the order with RandomSample is important because in the next step we are going to annotate the images one by one, but we're not going to annotate all of them, and we want to annotate images from all three datasets, not just the first one.

We build the following interface for annotating these images:

$i = 1;
$marked = {};
$unmarked = {};
$selected = {};
subimages = ImagePartition[First[imgs], {45, 30}];
Column[{
  Dynamic[
   Map[Button[#, AppendTo[$selected, #]] &, subimages, {2}] // Grid,
   TrackedSymbols :> {subimages}
   ],
  Button[
   "Next grid",
   $marked = Join[$marked, $selected];
   $unmarked = Join[$unmarked, Complement[Flatten[subimages], $selected]];
   $selected = {};
   subimages = ImagePartition[imgs[[++$i]], {45, 30}];
   ],
  Dynamic@StringJoin[ToString[$i], "/", ToString[Length[imgs]]]
  }]

This worked well enough. To use this, click on all the marked images and then hit the button at the bottom to go to the next image. In the end, $marked is a list of images of marked patches, and $unmarked is a list of images of unmarked patches.

I annotated one hundred images (834 marked patches) using this interface and exported the patches. You can download the dataset here. If you want to follow along, download and unzip the dataset, and then import the images like this:

unmarked = Import /@ FileNames[".../patches/unmarked/*.png"];
marked = Import /@ FileNames[".../patches/marked/*.png"];

marked = Thread[marked -> 1];
unmarked = Thread[unmarked -> 0];

train = Part[marked, ;; 500]~Join~Part[unmarked, ;; 500];
train = RandomSample[train];
test = Part[marked, 501 ;;]~Join~Part[unmarked, 501 ;; 834];
test = RandomSample[test];

I have more unmarked patches than marked ones, but I make the datasets such that I have the same number from each class. Having a balanced dataset is usually preferable.

Now we train a classifier, Mathematica will automatically select the type of classifier for us:

classifier = Classify[train];

Next, we test it on the test set:

Mean@N@Boole[test /. Rule[in_, out_] :> classifier[in] == out]

0.992515

Awesome, it worked for 99.25 percent of the images. We can look at the images it missed using this code:

Pick[test, test /. Rule[in_, out_] :> classifier[in] != out]

I'm not sure why it missed these because it looks like it shouldn't have, but it will probably be resolved if we annotate more images. In any case, this was just whatever classifier Mathematica decided to use for us, there might be a better one. In fact, setting Method to SupportVectorMachine we get a classifier which performs perfectly:

classifier = Classify[train, Method -> "SupportVectorMachine"];
Mean@N@Boole[test /. Rule[in_, out_] :> classifier[in] == out]

1

Locating objects

The classifier works great if we know where to look for the objects. But if we don't know how to extract the subimages to run the classifier on, then how can we do it? The most time-honored method for this is the sliding window technique. We simply take a window and slide it over the image, extracting subimages as we go along. We then run the classifier on each subimage to determine if it is a match or not.

imagePartition[{nrows_, ncols_}, {w_, h_}, {stepx_, stepy_}] := Table[{
    Rectangle[{c, nrows - r}, {c + w, nrows - r - h}],
    ImageTake[img, {r, r + h}, {c, c + w}]
    },
   {r, 1, nrows - h, stepy},
   {c, 1, ncols - w, stepx}
   ] // Flatten[#, 1] &

classify[classifier_, subimages_, thresh_] := Map[
  #[1] > thresh &,
  classifier[subimages, "Probabilities"]
  ]

{ncols, nrows} = ImageDimensions[img];
{boxes, subimages} = Transpose@imagePartition[{nrows, ncols}, {45, 30}, {10, 10}];
selected = Pick[boxes, classify[classifier, subimages, 0.5]];

HighlightImage[img, selected]

In the previous image, we see the subimages that the classifier thinks are marked patches. But there is a problem: the algorithm has selected several boxes for each patch. A solution to this problem is known as non-maximum suppression, and it works like this:

• Group all boxes together that overlap each other with a given fraction, such as fifty percent.
• Take the mean of the bottom left corners and the top right corners of all the boxes in each group to find a new box that will represent that group.

The following code implements this:

intersectingQ[r1_, r2_, thresh_] := Area@RegionIntersection[r1, r2] > thresh Area[r1]

meanRectangle[rectangles_] := Module[{bottomLeft, topRight},
  bottomLeft = rectangles[[All, 1]];
  topRight = rectangles[[All, 2]];
  Rectangle[Mean[bottomLeft], Mean[topRight]]
  ]

nonMaximumSuppression[boxes_, thresh_] := Module[{graph, groups},
  graph = AdjacencyGraph@Boole@Outer[intersectingQ[#, #2, thresh] &, bboxes, bboxes];
  groups = VertexList /@ ConnectedGraphComponents[graph];
  meanRectangle@Part[boxes, #] & /@ groups
  ]

HighlightImage[
 img,
 nonMaximumSuppression[selected, 0.5]
 ]

This worked quite well! We didn't miss any marked patches, and the boxes are reasonable. The positions of the boxes close to the border are biased inwards because we're taking the mean of all the proposed boxes, and they are all located on one side of the marked patch. This could probably be solved by padding. Also note that decreasing the step size used when generating the subimages will increase precision as well.