10
$\begingroup$

I'm hoping someone can help me improve the following fun code-snippet, which takes a target color and searches through a set of images to find the ones that most closely match:

data = ResourceData["CIFAR-10", "TestData"][[;; ;; 100, 1]];
cols = DominantColors[#, 10, {"Color", "Coverage"}] & /@ data;
ColorSlider[Dynamic[targetColor]]
Dynamic[distances = Table[Min[(Function[c, ColorDistance[c, targetColor]] /@ cc[[All, 1]])*(1/cc[[All, 2]])], {cc, cols}];
 Magnify@Row@data[[Ordering[distances, 5]]]
 ]

enter image description here

Ok, so it basically works. But here are the two things missing:

Problem 1. I’m hoping someone can help make it more efficient so it can scale a little better, the example runs on only 100 images. I would like to replace the brute-force Table with a call to Nearest - but I'm not quite sure how.

Problem 2. I'd also like to search based on multiple colors (2 or more weighted colors), but have not had much success. Here's the naive approach of adding the distances:

Column@{ColorSlider[Dynamic[targetColor1]], ColorSlider[Dynamic[targetColor2]]}
        Dynamic[distances = Table[Min[(Function[c, ColorDistance[c, targetColor1]] /@ cc[[All, 1]])*(1/cc[[All, 2]]) + (Function[c, ColorDistance[c, targetColor2]] /@ cc[[All, 1]])*(1/cc[[All, 2]])], {cc, cols}]; Magnify@Row@data[[Ordering[distances, 10]]]]

enter image description here

This doesn't work very well, I'm thinking of perhaps tweaking the distance function and using the LAB colorspace.

References & Links:

Helpful examples and related articles:

$\endgroup$
4
  • 2
    $\begingroup$ Some sort of dimension reduction sounds like it could be useful here. $\endgroup$
    – ktm
    Feb 5, 2019 at 4:52
  • 1
    $\begingroup$ A list of color coordinates with weights doesn’t seem super high dimensional though right? $\endgroup$
    – M.R.
    Feb 5, 2019 at 4:58
  • 1
    $\begingroup$ Your example with two query colors actually isn’t that bad $\endgroup$
    – user5601
    Feb 5, 2019 at 5:29
  • $\begingroup$ So it's not super terrible, the but colors sort of average out and most importantly - the lookup is totally inefficient! $\endgroup$
    – M.R.
    Feb 5, 2019 at 16:26

2 Answers 2

8
+50
$\begingroup$

An alternative approach might be to use the built-in ImageDistance function.

data = ResourceData["CIFAR-10", "TestData"][[;; ;; 100, 1]];
ColorSlider[Dynamic[targetColor]]
Dynamic[dist = ImageDistance[Image[targetColor], #] & /@ data;
        Magnify@Row@data[[Ordering[dist, 5]]]]

Of course you can choose any of the possible DistanceFunctions. This can be sped up significantly by using a NearestFunction to calculate the ImageDistance between the target and the collection of images. For example:

data = ResourceData["CIFAR-10", "TestData"][[;; ;; 100, 1]];
nf = Nearest[data];
ColorSlider[Dynamic[targetColor]]
Dynamic[nf[Image[targetColor], 5]]

To address the issue of a multiple color selector, this can be incorporated smoothly by setting the "targetColor" image to have two different colors. Now the ImageDistance will react to both the chosen colors.

data = ResourceData["CIFAR-10", "TestData"][[;; ;; 100, 1]];
nf = Nearest[data];
{ColorSlider[Dynamic[targetColor1]], ColorSlider[Dynamic[targetColor2]]}
 Dynamic[targetColorAll = 
   ImageAssemble[{Image[targetColor1], Image[targetColor2]}]]
 Dynamic[nf[Image[targetColorAll], 5]]
$\endgroup$
8
  • 1
    $\begingroup$ Earth mover is notoriously slow and won’t scale $\endgroup$
    – M.R.
    Feb 7, 2019 at 2:10
  • $\begingroup$ None of them are fast $\endgroup$
    – M.R.
    Feb 7, 2019 at 2:28
  • 1
    $\begingroup$ The only way to scale it is to compute color features and index them into a approximate nearest neighbors forrest $\endgroup$
    – M.R.
    Feb 7, 2019 at 2:48
  • $\begingroup$ Also this doesn’t address multicolor search $\endgroup$
    – M.R.
    Feb 7, 2019 at 4:26
  • $\begingroup$ See updates, which address both your issues, still within the framework of using ImageDistance. $\endgroup$
    – bill s
    Feb 7, 2019 at 16:26
7
$\begingroup$

Here's a NearestFunction that will do what you did with Table:

nf = Nearest[
  Hold /@ cols -> "Index", (* Hold seems to be necessary to make sure Nearest doesn't get confused by the inner lists *)
  DistanceFunction -> Function[
    Min @ Divide[
      Map[
       Function[col,
        ColorDistance[#1, col]
        ],
       #2[[1, All, 1]]
       ],
      #2[[1, All, 2]]
      ]
    ]
  ];

I also recommend that you use the TrackedSymbols -> {targetColor} option in your Dynamic to make sure it only updates whenever targetColor changes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.